Plunge-spectrum regularizer

Integrate this into a learned token-mixing, graph, or attention matrix that can be made self-adjoint and contractive. Use S = U @ diag(sigmoid(theta)) @ U.T, with U orthonormal, or construct an implicit symmetric operator followed by eigenvalue clipping to [0,1]. During training, add lambda_plunge * trace_estimate to the task loss. Estimate the trace with r Gaussian Hutchinson probes: for each probe z, repeatedly apply S and form v = v - S(v) after first setting v = S(z); equivalently compute w=(S-S@S)z and apply the same operator m-1 additional times, then accumulate dot(z,w). Use m=1 initially, since it requires two applications of S, and test m=2 for stronger suppression of mid-spectrum modes. For a chosen epsilon, report the empirical upper bound trace_estimate / (epsilon*(1-epsilon))**m; this follows directly from the paper's plunge inequality. At inference, threshold sigmoid(theta) at 1-epsilon and retain only the active eigenvectors, yielding a binary projector or low-rank approximation. The paper provides the spectral inequality and fixed-depth trace scaling; the implementation estimates the trace stochastically and does not assume the unknown O_m(1) constant. First test on CIFAR-10 with a 2-layer 14x14 ViT, comparing no regularizer, nuclear-norm regularization, and the plunge penalty at equal rank. Track accuracy, singular-value histograms, estimated plunge count, retained rank, quantized accuracy, and sensitivity to input perturbations. Success means fewer transition modes and smaller retained rank at unchanged accuracy.