The Normal Procrustes Problem: A Riemannian Optimization Approach
arXiv:2608.19513
2026
Architecture
1 ideas extracted · analyzed Sep 1, 2026
What the math gives to ML
The paper turns least-squares fitting of a matrix into a constrained problem over normal operators, defined by commuting with their adjoint: $A^*A=AA^*$. This suggests a spectral linear layer whose weight is parameterized as $A=U\operatorname{diag}(\lambda)U^*$, separating a unitary eigenbasis from learnable eigenvalues while guaranteeing normality and enabling direct control of the spectrum. The most practical transfer is to use this layer for stable recurrent or state-space transitions, with Procrustes fitting for initialization and Riemannian updates that preserve unitarity exactly.
Ideas from this paper
Unverified
2026
Replace an unconstrained dense transition or recurrent matrix with a normal matrix $A=U\operatorname{diag}(\lambda)U^*$, where $U$ is unitary and $\lambda$ contains learnable eigenvalues. The layer can be initialized by fitting a normal matrix to input-output pairs through the paper's objective, then trained with Riemannian updates that keep $U$ unitary and preserve the normal-operator structure.
Useful6/10
Difficulty6/10
Novelty6/10