MP Bulk Conditioning Regularizer

Insert the regularizer after the tensorized embedding and before its following linear projection. (1) For each minibatch, construct Z with rows z_i=phi(x_i), optionally center columns if the baseline layer uses centered features, and define C=Z^T Z/B. For large p, never materialize C. (2) Estimate lambda_max with power iteration: initialize v randomly, repeat v<-Cv/||Cv|| for K=5 iterations, and set lambda_max_hat=v^T C v. Estimate lambda_min with a short Lanczos routine; for a simpler approximation, choose a safe shift mu greater than the current lambda_max and power-iterate mu I-C, then recover lambda_min_hat=mu-v^T(mu I-C)v. (3) Compute c=p/m_eff, where m_eff is the minibatch size or a FIFO feature queue size. If c<1, set the lower target to a_c; if c>=1, set it to zero and retain only the upper-outlier penalty. Add L_MP to the task loss with alpha and beta initially between 1e-3 and 1e-2, use delta around 0.05 times the current median eigenvalue, and stop gradients through a_c and b_c. Use gradient accumulation or a feature queue so m_eff is not extremely small. (4) Log the estimated eigenvalues, MP edges, task loss, regularizer magnitude, and gradient norm. The first cheap experiment is a width-256 degree-2 tensor-feature MLP on CIFAR-10 or synthetic regression, comparing no regularizer, standard whitening, and L_MP at equal wall-clock budget. Success means fewer covariance outliers, a larger minimum-eigenvalue margin, smoother gradients, and equal or better validation accuracy. Failure is increased compute or suppression of task-relevant anisotropy without improved optimization.