Jacobi Moment Spectral Regularizer

Attach the regularizer to weight matrices of MLP expansion/projection layers or attention Q,K,V projections during training, initially excluding embeddings and output heads. Precompute target values mu_4, mu_6, and mu_8 from a 32-by-32 or 64-by-64 T_L for a selected q; these constants are detached from autograd. At each regularization step, form C implicitly. For each Hutchinson probe z, compute v_0=z and repeatedly apply v_{j+1}=W^T(Wv_j)/s, where s=tr(W^T W)/d is computed directly from the weight tensor. Estimate b_j by z^T v_j/d, average over K=1 or 2 fixed Rademacher probes, and add lambda times the squared deviations from mu_{2j}. Refresh probes every few hundred steps and backpropagate through W but not through the target moments. The paper contributes the deterministic recurrence and moment targets; trace estimation and coefficient selection are empirical. Begin with a six-layer width-256 MLP on CIFAR-10 or a two-layer transformer on a small language dataset. Compare no spectral regularization, Frobenius decay, spectral normalization, and this penalty at matched FLOPs. Track validation loss, gradient norm, estimated top singular value, and b_4,b_6,b_8. Success is reduced spectral-tail growth, improved validation performance, or stable training with a larger learning rate without divergence.