Spectral-Abscissa Early-Warning Scheduler
Implementation & benchmark of arXiv:2608.14978 — An Idealized Delay-Differential Model of Scuba Diver Porpoising and Runaway Ascent
Source paper: An Idealized Delay-Differential Model of Scuba Diver Porpoising and Runaway Ascent arXiv:2608.14978 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Use critical-slowing-down statistics from the delayed dynamical system to detect when training approaches an oscillatory instability. Rising lag-one autocorrelation and variance, together with a recovery-rate estimate approaching zero, trigger a learning-rate or momentum reduction before loss divergence occurs.
Formulas
Mathematical statement
Let y_n be a scalar training observable, such as minibatch loss residual, gradient norm, or a random parameter projection. Near a stable fixed point, approximate its slow mode by delta y_{n+1}=a delta y_n+epsilon_n, where epsilon_n is short-memory noise and a is the discrete dominant mode. If lambda is the continuous-time dominant eigenvalue and Delta t is the sampling interval, then a is approximately exp(alpha Delta t), where alpha=max_j Re(lambda_j) is the spectral abscissa. The lag-one autocorrelation rho_1 estimates a, the recovery rate is r=-log(rho_1)/Delta t, and the stationary variance is sigma_epsilon^2/(1-a^2). Approaching a Hopf boundary from the stable side makes alpha approach zero from below, so rho_1 approaches one, variance rises, and recovery rate falls.
Implementation notes
Add a monitoring callback to the training loop. Every M=10-50 optimizer steps, record a minibatch-loss residual after subtracting an exponential moving average, the global gradient norm, and one or more fixed random projections of the parameter update. On a rolling window of N=100-500 samples, compute variance and lag-one autocorrelation, then estimate the recovery rate as r_hat=-log(rho_1)/Delta t. Smooth each statistic with an exponential moving average and compare it with the median during the first 20% of training or with a no-delay control run. When rho_1 rises by 0.1-0.2, variance doubles, or r_hat falls below a fixed fraction of baseline, apply eta <- 0.5 eta and beta <- min(beta,0.9); restore hyperparameters only after r_hat recovers. The AR(1) relation and variance law are paper-derived; rho_1, variance, and r_hat are estimated empirically. First test on a small Transformer or ResNet with artificially stale gradients and compare fixed hyperparameters against this scheduler. The predicted signature is critical slowing down before divergence: rho_1 should increase toward one, variance should scale approximately as 1/(1-rho_1^2), and recovery rate should approach zero. After intervention, these trends should reverse. If no pre-divergence increase appears, reject the mechanism for that optimizer or change the observable.
Verification
Stage 1 · Toy mechanism gate: Failed ✗
Stage 2 · Mechanism transferred to benchmark: Not tested
Stage 2 · Practical benchmark result: Not run
Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built and numerically verified the AR(1) core: across a=0.2–0.98, lag-one autocorrelation RMSE was 0.00233, variance-law mean relative error was 0.73%, and recovery-rate RMSE was 0.00737. However, in the delayed-gradient quadratic experiment the scheduler never triggered before divergence for delays 2–12, so it provided no stability win over the fixed baseline; only the no-delay case remained stable for both.
- Agent confidence
- 9/10
- Baseline
- Delayed-gradient training diverged at steps 103, 97, 104, 122, 133, and 136 for delays 2, 4, 6, 8, 10, and 12 respectively; stable at delay 0.
- Idea
- Scheduler had identical divergence steps and outcomes for every tested delay; no trigger occurred before divergence. AR(1) mechanism checks: rho RMSE 0.00233, variance relative error 0.73%, recovery RMSE 0.00737.
Limitations: Only a synthetic scalar AR(1) verification and a 2D delayed quadratic were tested; no Transformer/ResNet, minibatch noise, CUDA training, genuine stale-gradient callback, oscillatory complex-mode system, or hyperparameter tuning of the monitoring window and thresholds was evaluated.
How to run: python3 experiment.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Artifacts
- experiment.py 3.2 KB View Raw
- report.md 1.4 KB View
- results.json 5.1 KB View Raw