Gauge-equivariant Adam for factored layers

(1) Integration point: implement this as an optimizer for any factored parameterization, beginning with matrix sensing or a LoRA linear layer W=W_0+UV^{ op}. Do not apply ordinary Adam independently to U and V. Maintain first moments mU and mV, but maintain one scalar second moment s for the complete pair of factors; decoupled weight decay can be applied separately and identically to U and V. (2) Pseudocode: initialize mU=mV=0 and s=0. At each step compute L=f(U@V.T), gradients gU,gV; update mU=beta1*mU+(1-beta1)*gU and mV=beta1*mV+(1-beta1)*gV; update s=beta2*s+(1-beta2)*(sum(gU*gU)+sum(gV*gV))/(numel(U)+numel(V)); bias-correct the moments; set U=U-eta*mUhat/(sqrt(shat)+epsilon) and V=V-eta*mVhat/(sqrt(shat)+epsilon). (3) The mathematical property to verify is exact gauge equivariance: sample an orthogonal Q, rotate U,V and their gradients, and compare the rotated original update with the update recomputed after rotation. The scalar s is invariant because Frobenius norms are right-orthogonally invariant. A stronger variant can use P_U=(UU^{ op}+VV^{ op}+lambda I)^{-1/2} and an analogous P_V, approximated with diagonal or low-rank methods. (4) First cheap experiment: use underdetermined matrix sensing with planted rank-r W_true, factor width k>r, and Gaussian measurements. Compare Adam, SGD, shared-scalar Adam, and Muon at matched FLOPs. Track recovery error, effective rank of UV^{ op}, training loss, and rotated-initialization consistency. The expected signal is lower final recovery error and smaller effective rank for shared-scalar Adam than Adam at matched training loss, while equivalent rotated initializations produce nearly identical trajectories. A failure criterion is no improvement over Adam or an equivariance error above 1e-5 in float32.