arXiv:2608.06218 2026 Optimization 1 ideas extracted · analyzed Aug 31, 2026
The paper provides a matrix-level optimization primitive whose exact solution is the negative polar factor of a momentum matrix. Unlike coordinate-wise updates, this direction solves a spectral-norm constrained linear minimization and treats all nonzero singular directions uniformly. The transferable opportunity is an optimizer for Stiefel-constrained neural-network weights: project momentum into the tangent space, apply the polar-factor direction, and retract by a polar decomposition so orthogonality is maintained exactly. This can be tested against AdamW with orthogonality penalties, ordinary Muon, and QR-based manifold updates.
Use the negative polar factor of a tangent-projected momentum matrix as the update direction for a weight matrix constrained to the Stiefel manifold. After taking the step, apply a polar retraction so the columns remain exactly orthonormal, avoiding penalty losses and constraint drift.