Multiplier-Bootstrap Spike Detector

(1) Integration point: insert a no-gradient diagnostic after a chosen Transformer MLP, CNN block, or residual stream. Collect the last n centered activation vectors z_i in R^d from a rolling calibration buffer; do not run the diagnostic on every minibatch. If the detector fires, modify the next forward pass with a low-rank covariance shrinker or whitening transform. (2) Pseudocode: compute mu=n^(-1) sum_i z_i and replace z_i by z_i-mu; compute the top eigenpair (lambda_1,u_1) of C by two or three power iterations; for b=1,...,B, sample independent Rademacher xi_i^(b), form the weighted covariance C*^(b)=n^(-1) sum_i(1+xi_i^(b))z_i z_i^T, and estimate its top eigenvalue by power iteration; set q to the (1-delta) empirical quantile and trigger when lambda_1(C)>q for two consecutive buffers. On trigger, use P=I-(1-rho)u_1u_1^T and replace z by Pz, or shrink only the spike via z_new=(I-u_1u_1^T)z+rho u_1u_1^T z with 0<rho<1. (3) Computed from the paper: the spectral-outlier principle, eigenvalue separation, and resolvent interpretation; estimated empirically: q, the aspect ratio d/n, and spike direction u_1. A resolvent variant estimates a bulk matrix Q after removing u_1 and searches for sign changes of M(lambda)=1+y^T(Q-lambda I)^(-1)y. (4) First experiment: train a ViT-Tiny or 6-layer Transformer on CIFAR-10 or WikiText-2, comparing baseline LayerNorm, always-on whitening, covariance shrinkage, and this gated detector at equal training FLOPs. Use B=32, delta=0.95, buffer size n=256, and trigger hysteresis. Measure validation loss, activation condition number, gradient norm spikes, and accuracy under distribution shift. Success means lower instability and equal-or-better validation accuracy without the degradation caused by always-on whitening.