A Continuous-Time Analysis of Smoothed Matrix-Polar Spectral Gradient Flows for Muon-Type Optimization

What the math gives to ML

The paper supplies a smooth spectral replacement for the matrix polar factor, preserving the singular-vector geometry of Muon while removing its singularity at rank-deficient momentum matrices. The key transferable asset is the bounded singular-value response \(\sigma \mapsto \sigma/\sqrt{\sigma^2+\epsilon}\): large singular modes receive approximately unit polar normalization, while small or noisy modes are attenuated continuously instead of being amplified by an ill-conditioned inverse square root. This suggests a drop-in matrix optimizer for transformer and MLP weight updates, with \(\epsilon\) scheduled or tuned to control noise sensitivity and numerical stability. The spectral potential \(\Phi_\epsilon\) also gives an explicit energy whose gradient is exactly the update direction, enabling optimizer diagnostics based on spectral dissipation.