Sign compression for Muon: SignMuon, MuonSign, and the Limits of Error Feedback
arXiv:2607.29674
2026
Optimization
2 ideas extracted · analyzed Aug 31, 2026
What the math gives to ML
The paper exposes a practically important mismatch between communication compression and Muon's matrix-valued Linear Minimization Oracle (LMO): elementwise sign compression after the polar/LMO step can ascend even on a linear objective, and error feedback on the Muon output does not universally repair it. The transferable asset is the distinction between compressing the gradient before the LMO, where error feedback can recover a standard nonconvex rate, and compressing the LMO output, which is empirically strong but lacks a general descent guarantee. A useful neural-network implementation is therefore a gradient-side error-feedback Muon optimizer, optionally augmented with a cheap descent-alignment safeguard that rejects the empirically attractive post-LMO sign update when it points uphill.
Ideas from this paper
△ Mechanism confirmed, baseline not beaten
2026
Compress the matrix gradient or momentum before applying Muon's polar LMO, and maintain an error residual in the uncompressed gradient space. The residual prevents systematic sign quantization bias from accumulating, unlike error feedback applied after the nonlinear polar/sign operation. This is suitable for distributed training because workers communicate one sign bit per matrix entry while the server still applies a matrix-aware Muon step.
Useful8/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Keep the empirically effective post-LMO sign update, but reject it whenever a fresh minibatch estimates that it is poorly aligned with the gradient. Fall back to the gradient-side error-feedback candidate in those cases. This converts the paper's constructive divergence warning into an inexpensive runtime safeguard rather than assuming that any sign placement is universally safe.
Useful7/10
Difficulty4/10
Novelty7/10