Schatten-Stable Noncommutative Matrix Layer

Integrate the layer after a channel projection in a small MLP, graph neural network, or vision block. Given a batch of feature vectors \(h\in\mathbb R^d\), construct two real symmetric matrices from learned projections, for example \(A=\tfrac12(M_A+M_A^T)\) and \(B=\tfrac12(M_B+M_B^T)\), with \(M_A=W_Ahh^TW_A^T\) and \(M_B=W_Bhh^TW_B^T\), or construct one matrix per sample from covariance statistics over tokens. Keep matrix dimension \(m\leq32\) for the MVP. The forward pseudocode is: (1) symmetrize \(A,B\); (2) compute eigendecompositions \(A=U\operatorname{diag}(a)U^T\) and \(B=V\operatorname{diag}(b)V^T\); (3) evaluate a smooth scalar function \(\varphi_\theta(x,y)\), initially a low-degree polynomial or two-dimensional spline; (4) form \(C_{ij}=\varphi_\theta(a_i,b_j)(U^TV)_{ij}\); (5) return \(Y=UCV^T\), followed by a learned linear projection and residual connection. Add the normalized commutator term from the displayed loss to the task loss. The DOI and eigendecompositions are computed exactly; the commutator penalty is exact; the property to validate empirically is improved perturbation stability. Estimate sensitivity by adding Gaussian noise \(\Delta A,\Delta B\) and measuring \(\|\mathcal L(A+\Delta A,B+\Delta B)-\mathcal L(A,B)\|_{S_2}\) against input perturbation size. First compare a 2-layer baseline MLP, an unconstrained matrix-function layer, and this layer on CIFAR-10 using 16- or 32-dimensional matrices, matching parameters and FLOPs as closely as possible. Success means lower output sensitivity and fewer exploding-loss runs under noisy inputs with equal or better validation accuracy; target at least 20% lower normalized output perturbation at the same clean accuracy. Ablate the commutator penalty, the smoothness of \(\varphi\), and matrix size, since excessive penalty may force near-commutativity and remove the intended benefit.