✓✓ Beats tuned baseline 2026

Spectral-Width Prethermal Training Schedule

Implementation & benchmark of arXiv:2607.12355 — From stable periodic orbits to many-body chaos: doubly tunable prethermalization via engineering of an emergent band structure

Usefulness7/10

Difficulty5/10

Novelty8/10

Training Stability Speedup Generalization Training dynamics Scheduler Optimizer Rnn Ssm Dynamical systems Spectral theory Stochastic processes Control theory

Source paper: From stable periodic orbits to many-body chaos: doubly tunable prethermalization via engineering of an emergent band structure arXiv:2607.12355 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use the paper's lifetime law as a controller for training or rollout difficulty. Estimate the active perturbation bandwidth R of hidden states or forecast errors and reduce the residual gain, increase the dispersion order W, or inject controlled bandwidth whenever the estimated prethermal lifetime becomes too short.

Formulas

$$P_t(k)=\frac{|\widehat{u_t}(k)|^2}{\sum_{k'}|\widehat{u_t}(k')|^2},\qquad R_t^2=\sum_k d(k,k_0)^2P_t(k).$$

$$\Delta\omega(k_0+q)\simeq a_W|q|^W\quad\Longrightarrow\quad \tau_t\simeq C R_t^{-W}.$$

$$R_{\max}=\left(\frac{C}{\tau_\star}\right)^{1/W},\qquad \eta_{t+1}=\operatorname{clip}\left[\eta_t\left(\frac{\tau_t}{\tau_\star}\right)^\alpha,\eta_{\min},\eta_{\max}\right].$$

Mathematical statement

Let u_t denote hidden-state perturbations, gradient noise, or forecast residuals, and let P_t(k) be their normalized Fourier power spectrum. Around a selected gapless point k_0, define R_t by the second spectral moment R_t^2=sum_k d(k,k_0)^2P_t(k), where d is wrapped frequency distance. If the architecture has dispersion Delta omega(k_0+q) approximately a_W|q|^W, the paper predicts tau_t=C R_t^{-W}, where C depends on coefficient scale and noise amplitude. A target lifetime tau_star implies R_t should remain below R_max=(C/tau_star)^{1/W}. A feedback controller can update the residual gain eta using eta_{t+1}=clip(eta_t(tau_t/tau_star)^alpha,eta_min,eta_max), with alpha>0, lowering eta when the estimated lifetime is too short. Hysteresis should be used to prevent minibatch noise from causing rapid switching.

Implementation notes

Integrate the controller after each optimizer step or every K training steps in a sequence model with the flat-band residual block, a state-space model, or a recurrent world model. Collect a minibatch of hidden states, residuals, or one-step forecast errors over a short temporal window, subtract the batch mean, compute an FFT along the temporal axis, and estimate P_t(k) and R_t. During a warmup phase, run fixed-bandwidth perturbation tests and fit log tau against log R to estimate C; W is computed from the architecture's Fourier symbol or estimated by a local log-log fit. During training calculate tau_hat=C max(R_t,epsilon)^{-W}. If tau_hat falls below tau_low, lower eta with the multiplicative rule, strengthen gradient clipping, or switch from a W=1 block to a preinitialized W=3 block. Restore eta only after tau_hat exceeds tau_high for several checks. Smooth R_t exponentially to avoid minibatch reactions. The first cheap experiment is a small GRU or S4-style model on adding, copy-memory, and sequential MNIST, comparing fixed learning rate, cosine decay, and this controller at equal compute. Record long-horizon loss, gradient norms, spectral width, and intervention times. The prediction is that widening the perturbation spectrum by factor b reduces retention time by approximately b^W; fitted log-time versus log-bandwidth should have slope -W, and controller interventions should cluster near the independently measured instability boundary.

Verification

Beats tuned baseline

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significant win ✓

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

How to read this: The implementation beat the baseline, but the claimed mechanism did not reproduce at benchmark scale. The gain may come from another effect; treat the cause as unresolved.

Stage 1 — Mechanism check agent confidence 8/10

Built a NumPy MVP implementing wrapped FFT spectral-width estimation, the lifetime controller, smoothing, hysteresis, and reproducible toy sweeps. The mechanism predictions were confirmed: log(tau)-log(R) slopes were -1.00, -2.00, and -3.00 for W=1,2,3; bandwidth multipliers matched b^W with relative error below 5e-16; and the isolated FFT estimator had the predicted inverse-scale slope -1.00. In the noisy controller toy, feedback reduced final cumulative growth from 1.4752 to 1.1607 and reduced mean width from 0.0900 to 0.0565, although the minimum transient lifetime was not improved.

Agent confidence: 8/10

Baseline: Fixed gain: final_loss=1.4752, mean_width=0.0900, mean_tau=617.3, interventions=0
Idea: Hysteretic spectral controller: final_loss=1.1607, mean_width=0.0565, mean_tau=1647.3, interventions=8, eta_final=0.668

Limitations: This is an analytical toy growth model, not a trained GRU/SSM or real neural-network benchmark. The lifetime constant is known rather than fitted online, compute/FLOP overhead was not measured, and only one fixed random seed and one noisy controller trajectory were compared.

How to run: python3 spectral_width_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.12355) Sep 2, 2026

Beats the tuned baseline by -0.0003 metric points (p=0.0081, wins 8 / 8 paired seeds; metric: lower is better).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0
Idea mean: 0
Effect (Δ): -0.0003 (n/a; negative = idea better)
Wins: 8 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Not confirmed ✗
Practical effect: Helps
Baseline tuning: swept over 3 configs

Limitations:

Only the built-in dynamics track was tested. The controller used a heuristic lifetime constant rather than fitting C online, the neural-scale R^-2 mechanism prediction was not confirmed, and wall-clock/FLOP overhead was not separately measured.

How to run: python3 bench_spectral_schedule.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Worked ✓

Agent confidence: 8/10

Implemented the spectral-width hysteretic training schedule on the structurally matched dynamics track using identical rnn_small models and paired seeds. Across 8 seeds, the controller achieved lower test MSE on every pair: mean 0.0003913901 versus 0.0007366434 for the cosine-decay baseline, with paired delta -0.0003452533 and permutation p=0.0081. The neural-scale mechanism signature did not confirm the predicted R^-2 law, so the observed win is empirical rather than mechanistically validated.

Baseline: dynamics/rnn_small, cosine-decay Adam, best lr=0.006; mean test MSE=0.0007366434
Idea: Hysteretic spectral-width controller, best lr=0.006; mean test MSE=0.0003913901; paired delta=-0.0003452533; 8/8 wins; permutation p=0.0081

Benchmark result