# Mpemba Mode-Filtered Training - ID: 1222 - Canonical URL: https://synthcore.org/idea/1222/mpemba-mode-filtered-training - API JSON: https://synthcore.org/api/idea/1222.json - API Markdown: https://synthcore.org/api/idea/1222.md - Verification status: mech_ok_no_baseline - Source: [arXiv:2607.21226](https://arxiv.org/abs/2607.21226) - Category: dynamics - Solves: stability, speedup, generalization - ML areas: optimizer, training-dynamics, regularization - Math tags: dynamical-systems, spectral-theory, information-theory, control-theory - Ratings: usefulness 7/10; difficulty 6/10; novelty 8/10 ## Idea description Track a symmetry-asymmetry functional of network outputs or hidden states under a chosen transformation or channel and estimate which relaxation modes control its late-time decay. Use a short warm-up trajectory to suppress the slow asymmetry mode while allowing larger initial asymmetry in faster modes, producing a training trajectory that can overtake a nominally better-initialized trajectory. This transfers the paper's quantum Mpemba effect as a mode-overlap principle rather than requiring an exact Kramers-Wannier duality. ## Mathematical statement Let T be a stochastic or linear symmetry channel acting on predictions or representations, with invariant states satisfying T p=p. For a model state p_t, define asymmetry A(p_t)=D_KL(p_t||T p_t), or use the squared Euclidean version A_2(p_t)=1/2||p_t-Tp_t||_2^2 when probabilities are unavailable. Near an invariant state, let the training or inference dynamics be dz/dt=-Lz, where z is the symmetry-breaking component, L v_k=lambda_k v_k, and 0A_b(0) but |c_a1|<|c_b1|. In a two-mode approximation, the crossing time is t_*=[2(lambda_2-lambda_1)]^{-1} log[(c_a2^2-c_b2^2)/(c_b1^2-c_a1^2)], provided the logarithm argument is positive. The late-time slope is d log A_2/dt -> -2 lambda_j, where j is the smallest index with c_j nonzero. ## Key formulas - $$A(p)=D_{\mathrm{KL}}(p\|Tp)=\sum_i p_i\log\frac{p_i}{(Tp)_i},\qquad A_2(z)=\frac12\|z-Tz\|_2^2.$$ - $$\dot z=-Lz,\quad Lv_k=\lambda_kv_k,\quad z(t)=\sum_k c_k e^{-\lambda_k t}v_k,\quad A_2(t)=\frac12\sum_k c_k^2e^{-2\lambda_k t}.$$ - $$A_a(0)>A_b(0),\quad |c_{a1}|<|c_{b1}|\ \Longrightarrow\ \text{possible Mpemba crossing}.$$ - $$t_*=\frac{1}{2(\lambda_2-\lambda_1)}\log\left(\frac{c_{a2}^2-c_{b2}^2}{c_{b1}^2-c_{a1}^2}\right),\qquad \frac{d}{dt}\log A_2(t)\to-2\lambda_j.$$ ## Implementation notes Integrate this at the end of each training step or at selected checkpoints. Choose a transformation T that should leave the task invariant: for images use a reflection or rotation with class-preserving labels; for a transformer use token renaming or sequence reversal when semantically valid; for a graph network use node relabeling. Run the model on an input batch and its transformed batch, then form the symmetry-breaking vector z=(q_theta(x)-Tq_theta(x))/sqrt(B), where q_theta is the logits or pooled hidden representation and T acts on output coordinates. Compute A_2=0.5||z||^2 and record it without backpropagation. During a 50-200 step probe, estimate a low-rank relaxation operator by fitting z_{t+1}=M z_t with ridge regression, then compute lambda_k=-log|eig_k(M)|/Delta t. Verify the fitted modes by comparing projected amplitudes c_k=v_k^T z_t with observed trajectories. Construct a mode-filtered update by adding a temporary control term -gamma v_1 v_1^T z to the representation or logit update, or by projecting the symmetry-breaking gradient with I-v_1v_1^T. Remove this control after the probe. The paper supplies the mechanism that restoration depends on slow-mode overlap; the spectrum and coefficients are estimated empirically. First experiment: train a small MLP on MNIST with reflection-invariant labels, comparing standard Adam, an A_2 penalty, and mode filtering. Use equal final compute and deliberately create pairs where the filtered run has larger initial A_2. The predicted signature is a crossing only when the larger-asymmetry run has smaller estimated |c_1|; measured crossing time should match the two-mode formula within about 30%, and its late log-asymmetry slope should approach -2 lambda_2 rather than -2 lambda_1. For discrete control updates z_{n+1}=(I-eta L-gamma v_1v_1^T)z_n, stability requires |1-eta lambda_k-gamma delta_{k1}|<1 for every retained mode. ## Verification - Status: mech_ok_no_baseline - Mechanism evidence: yes - Mechanism confirmed: yes - Practical verdict: no_effect - Verdict: Built a reproducible two-mode Mpemba mode-filtering experiment in mpemba_experiment.py. The mechanism manifested: discrete crossing-time predictions matched observed crossings within 0.07–0.67%, the stability boundary was predicted at gamma=1.8 and divergence began above it, and suppressing the slow mode changed the late log-A2 slope from -0.1026 to -0.2671 as predicted. The filtered quadratic-training trajectory started with larger A2 (0.52 vs 0.20) yet crossed below baseline at step 2 and ended much lower, although no real MLP/MNIST experiment was run, so this is not evidence of a practical neural-network win. ### Mechanism check - Verdict: Built a reproducible two-mode Mpemba mode-filtering experiment in mpemba_experiment.py. The mechanism manifested: discrete crossing-time predictions matched observed crossings within 0.07–0.67%, the stability boundary was predicted at gamma=1.8 and divergence began above it, and suppressing the slow mode changed the late log-A2 slope from -0.1026 to -0.2671 as predicted. The filtered quadratic-training trajectory started with larger A2 (0.52 vs 0.20) yet crossed below baseline at step 2 and ended much lower, although no real MLP/MNIST experiment was run, so this is not evidence of a practical neural-network win. - Confidence: 9/10 - Limitations: Only an exactly solvable two-mode linear/quadratic toy system was tested. No MNIST, neural-network optimizer, learned mode estimation, reflection transformation, generalization metric, wall-clock comparison, or robustness to noise/nonlinear training was evaluated. ### Practical benchmark - Paired seeds: 8 - Baseline mean: 0 - Idea mean: 0 - p-value: 0.79715 - Paired wins: 4/8 - Benchmark verdict: no measurable effect ## Artifacts - [bench_report.json](https://synthcore.org/code/494/bench_report.json) - [mpemba_bench.py](https://synthcore.org/code/494/mpemba_bench.py) - [mpemba_experiment.py](https://synthcore.org/code/494/mpemba_experiment.py) - [report.md](https://synthcore.org/code/494/report.md) - [report_bench_2026-08-31T155426.md](https://synthcore.org/code/494/report_bench_2026-08-31T155426.md) - [results.json](https://synthcore.org/code/494/results.json) - [Download all files as ZIP](https://synthcore.org/download/494) ## Disclaimer AI-generated research hypothesis, automatically tested. Not peer-reviewed.