Gumbel escape-time controller
Implementation & benchmark of arXiv:2608.30624 — Time-delayed feedback turns Arrhenius escape logarithmic
Source paper: Time-delayed feedback turns Arrhenius escape logarithmic arXiv:2608.30624 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Use the paper's extreme-value escape statistics as a diagnostic for delayed-gradient bursts. If many stochastic minibatch realizations escape through an unstable delay mode, their first-passage times should become approximately Gumbel distributed, allowing the optimizer to distinguish useful basin escape from destructive divergence and to terminate or retune the burst automatically.
Formulas
Mathematical statement
After crossing the delay bifurcation, let r greater than zero be the dominant unstable growth rate and A the stochastic seed amplitude. Escape to radius R occurs at T=(log R-log A) divided by r. When the relevant seed or amplified fluctuation is an extreme-value variable, the centered escape time has Gumbel cumulative distribution F(T)=exp(-exp(-(T-mu)/beta)), where mu is the location parameter and beta is the scale parameter. The linear growth model predicts that beta is proportional to 1 divided by r. Therefore the controller should observe a transition from mostly nonescaping trajectories below the delay threshold to Gumbel-like first-passage times above it, with rescaled distributions approximately collapsing when time is multiplied by r.
Implementation notes
Attach the monitor to an optimizer that uses stale gradients, delayed target parameters, or any deliberately delayed update. At the beginning of each delayed burst, define an escape event as the first step at which normalized displacement q at time t equals the parameter distance from the burst start divided by the square root of the estimated gradient-noise trace plus epsilon, and q exceeds a chosen radius R. Run 16 to 32 inexpensive replicas with different minibatch orderings or random seeds, recording each first-passage time T. Fit the Gumbel location mu and scale beta by maximum likelihood, and estimate r from the slope of log(q) during the approximately linear amplification phase. Permit another short burst only if a goodness-of-fit test is acceptable, for example Kolmogorov-Smirnov p greater than 0.05, beta times r is stable across bursts, and escaped replicas improve validation loss. If displacement grows without successful escape, reduce tau toward 0.8 tau c, lower the learning rate, or terminate the burst. The paper provides the logarithmic amplification and Gumbel mechanism; estimate r, mu, beta, effective curvature k, and noise scale empirically. First test a two-dimensional noisy double-well system, followed by a small MNIST MLP. Sweep delays around tau c=pi divided by 2k and collect at least 32 passage times per setting. The falsifiable prediction is that below threshold most runs do not escape within the observation window, while above threshold a Gumbel fit improves over an exponential fit and beta times r remains approximately constant, within 30 percent, across delay settings.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗
Stage 2 · Practical benchmark result: Significantly worse ✗
Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built and numerically verified the Gumbel escape-time mechanism in a delayed linear toy model. The predicted delay threshold matched to 2.3e-13 relative error, beta·r stayed within about 5% of its predicted constant, Gumbel fits strongly beat exponential fits, and logarithmic boundary scaling matched within 1.2%. This validates the mechanism, but not yet an optimizer or MNIST training win.
- Agent confidence
- 9/10
- Baseline
- Exponential escape fit: consistently much worse AIC; no optimizer/MNIST baseline was run.
- Idea
- Threshold tau_c=1.5707963 observed numerically; beta*r={0.996,1.005,0.964,1.049}; log-boundary slope=2.248 versus predicted 2.222; all Gumbel KS p-values exceeded 0.05.
Limitations: The experiment used explicit extreme-value seeds rather than arbitrary minibatch noise. No true noisy double-well integration, stale-gradient optimizer, MNIST MLP, GPU benchmark, or measured validation-loss improvement was tested; the controller's validation signal is externally supplied.
How to run: python3 experiment.py
Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.30624)
Significantly WORSE than the tuned baseline (+143.8%, p=0.0081, wins 0 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 0.0004
- Idea mean
- 0.0009
- Effect (Δ)
- 0.0005 (+143.8%; negative = idea better)
- Wins
- 0 / 8 paired seeds
- p-value
- 0.0081 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±44.0%
- Mechanism
- Not confirmed ✗
- Practical effect
- Hurts
- Baseline tuning
- swept over 3 configs
Only the built-in dynamics track was tested; no custom delayed-gradient task, vision/tabular/sequence transfer, longer burst schedules, or alternate escape thresholds were evaluated. The controller's online monitor did not observe unstable escapes in this stable benchmark, so the Gumbel fit could not be tested at neural-network scale.
How to run: python3 bench_gumbel_controller.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Stage 2 — Benchmark latest Failed ✗
Agent confidence: 10/10
Built a local Gumbel escape-time controller as a delayed-gradient training intervention on the structurally matched dynamics track with rnn_small, using 8 paired seeds, a baseline learning-rate sweep, and a parity-compliant three-setting idea sweep. Tuned Adam achieved test MSE 0.0003815 versus 0.0009299 for the selected controller, with paired delta +0.0005484 and permutation p=0.0081; the idea was significantly worse. No trained-model escape events occurred, so beta·r could not be estimated and the mechanism signature was not confirmed.
- Baseline
- dynamics/rnn_small tuned Adam, best lr=0.01; 8-seed mean test MSE=0.0003814903
- Idea
- Gumbel delayed-gradient controller, selected lr=0.003 delay=1; 8-seed mean test MSE=0.0009298879; paired delta=+0.0005483976; permutation p=0.0081; 0 escape events
Benchmark result
Significantly WORSE than the tuned baseline (+143.8%, p=0.0081, wins 0 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 0.0004
- Idea mean
- 0.0009
- Effect (Δ)
- 0.0005 (+143.8%; negative = idea better)
- Wins
- 0 / 8 paired seeds
- p-value
- 0.0081 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±44.0%
- Mechanism
- Not confirmed ✗
- Practical effect
- Hurts
- Baseline tuning
- swept over 3 configs
Only the built-in dynamics track was tested; no custom delayed-gradient task, vision/tabular/sequence transfer, longer burst schedules, or alternate escape thresholds were evaluated. The controller's online monitor did not observe unstable escapes in this stable benchmark, so the Gumbel fit could not be tested at neural-network scale.
How to run: python3 bench_gumbel_controller.py
Artifacts
- REPORT.md 1.8 KB View
- bench_gumbel_controller.py 6.3 KB View Raw
- bench_report.json 7.5 KB View Raw
- experiment.py 4.3 KB View Raw
- report.md 1.2 KB View
- report_bench_2026-09-02T014500.md 4.3 KB View
- results.json 3.5 KB View Raw