Mpemba Mode-Filtered Training

Integrate this at the end of each training step or at selected checkpoints. Choose a transformation T that should leave the task invariant: for images use a reflection or rotation with class-preserving labels; for a transformer use token renaming or sequence reversal when semantically valid; for a graph network use node relabeling. Run the model on an input batch and its transformed batch, then form the symmetry-breaking vector z=(q_theta(x)-Tq_theta(x))/sqrt(B), where q_theta is the logits or pooled hidden representation and T acts on output coordinates. Compute A_2=0.5||z||^2 and record it without backpropagation. During a 50-200 step probe, estimate a low-rank relaxation operator by fitting z_{t+1}=M z_t with ridge regression, then compute lambda_k=-log|eig_k(M)|/Delta t. Verify the fitted modes by comparing projected amplitudes c_k=v_k^T z_t with observed trajectories. Construct a mode-filtered update by adding a temporary control term -gamma v_1 v_1^T z to the representation or logit update, or by projecting the symmetry-breaking gradient with I-v_1v_1^T. Remove this control after the probe. The paper supplies the mechanism that restoration depends on slow-mode overlap; the spectrum and coefficients are estimated empirically. First experiment: train a small MLP on MNIST with reflection-invariant labels, comparing standard Adam, an A_2 penalty, and mode filtering. Use equal final compute and deliberately create pairs where the filtered run has larger initial A_2. The predicted signature is a crossing only when the larger-asymmetry run has smaller estimated |c_1|; measured crossing time should match the two-mode formula within about 30%, and its late log-asymmetry slope should approach -2 lambda_2 rather than -2 lambda_1. For discrete control updates z_{n+1}=(I-eta L-gamma v_1v_1^T)z_n, stability requires |1-eta lambda_k-gamma delta_{k1}|<1 for every retained mode.