✓✓ Beats tuned baseline 2026

Escape-Threshold Learning-Rate Controller

Implementation & benchmark of arXiv:2608.02933 — From Flows to Maps: Sampling Laws for Attractor Intensity and Bounded-Noise Escape

Usefulness7/10

Difficulty6/10

Novelty9/10

Dynamics Stability Speedup Optimizer Scheduler Training dynamics Rnn Dynamical systems Control theory Stochastic processes

Source paper: From Flows to Maps: Sampling Laws for Attractor Intensity and Bounded-Noise Escape arXiv:2608.02933 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use bounded-noise escape as a measurable stability transition to adapt the learning rate or recurrent integration step before catastrophic loss of confinement. Periodically estimate the disturbance radius at which the current training dynamics exits its stable region, then adjust the step size to maintain a fixed safety margin.

Formulas

$$x_{k+1}=F_h(x_k)+\delta_k,\quad \|\delta_k\|\leq r$$

$$r>\mu_h(A)\Longrightarrow\text{loss of controlled confinement under the finite-exit condition}$$

$$\rho_h=\frac{\sigma h}{\widehat{\mu}_h}$$

$$h_{new}=h\,\mathrm{clip}\left[\left(\frac{\rho_{star}}{\rho_h}\right)^\alpha,c_{min},c_{max}\right]$$

Mathematical statement

For a discrete neural dynamical system x_(k+1) = F_h(x_k) + δ_k, let A be the desired attractor or late-training region and μ_h(A) the largest kick amplitude preserving controlled confinement in its basin. If the persistent disturbance amplitude exceeds μ_h(A), finite escape is expected under the paper's finite-exit condition. The normalized quantity μ_h(A)/h estimates a continuous-time disturbance-rate threshold. Let σ be the measured natural disturbance amplitude, such as a high percentile of minibatch-gradient variation. The dimensionless safety ratio is ρ_h = σh/μ_h(A). A controller can maintain ρ_h near a target ρ_star below one. The sampling theorem predicts how this threshold changes with h through the Lipschitz bounds.

Implementation notes

Attach the controller to the learning-rate scheduler of SGD, Adam without its own changing step scale, a neural ODE solver, or an RNN/SSM integration step. Every T updates, freeze the current parameters and run M short probe trajectories from recent states. Add bounded random or worst-direction perturbations with radius r. Estimate σ as the 95th percentile of naturally occurring minibatch-gradient differences, normalized into parameter-update units. Binary-search the largest probe radius μ̂_h that remains inside a region B for K steps; use hysteresis and require q consecutive boundary violations before declaring escape. Compute ρ_h = σh/μ̂_h and update h with the displayed multiplicative rule, restricting changes by c_min and c_max. The paper supplies the safety-threshold interpretation and sampling normalization; B, σ, the escape test, and μ̂_h are empirical. The first cheap experiment is CIFAR-10 with a small ResNet and SGD, comparing cosine decay, gradient clipping, and this controller under injected bounded minibatch noise. The predicted signature is a sharp increase in escape frequency when ρ_h crosses one, and approximately invariant escape behavior when h changes while μ̂_h/h is held fixed. Measured thresholds should satisfy the sampling bounds within about 20 percent.

Verification

Beats tuned baseline

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significant win ✓

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

How to read this: The implementation beat the baseline, but the claimed mechanism did not reproduce at benchmark scale. The gain may come from another effect; treat the cause as unresolved.

Stage 1 — Mechanism check agent confidence 9/10

Built a readable toy verification for the escape-threshold controller on the exactly solvable stable flow x'=-lambda*x with A=[-1,1]. The predicted escape transition r_c=(1-exp(-lambda h))/h matched binary-search estimates to below 1e-15 relative error at h=0.05–1.0, and every observed normalized threshold satisfied the paper’s sampling bounds; the controller also moved rho toward the 0.70 target and improved noisy-quadratic final mean loss from 0.09799 to 0.01476 versus fixed-step SGD. This is a clear mechanism signal, but it is not evidence of a CIFAR-scale win.

Agent confidence: 9/10

Baseline: Fixed h=0.9: safety-ratio regime means [1.269, 0.684, 1.074]; noisy-quadratic final mean loss 0.09799.
Idea: Adaptive h: safety-ratio last-5 means [0.794, 0.619, 0.741] across high/low/medium noise; noisy-quadratic final mean loss 0.01476, final h=0.0613; threshold relative error <1e-15 and all 6 sampling-bound checks passed.

Limitations: Only a one-dimensional linear attractor and scalar noisy quadratic were tested; the probe uses the exact threshold rather than estimating it from finite neural-network trajectories, and no CIFAR-10, ResNet, Adam, gradient-variation estimator, wall-clock, or multi-dimensional nonlinear stability experiment was run.

How to run: python3 escape_threshold_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.02933) Sep 2, 2026

Beats the tuned baseline by 81.2% (p=0.0081, wins 8 / 8 paired seeds; metric: lower is better).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.1824
Idea mean: 0.0342
Effect (Δ): -0.1482 (−81.2%; negative = idea better)
Wins: 8 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±14.6%
Mechanism: Not confirmed ✗
Practical effect: Helps
Baseline tuning: swept over 3 configs

Limitations:

Only the registered dynamics track was tested. The probe is a compact local parameter-direction estimate rather than the full M-trajectory, K-step binary-search escape protocol. Wall-clock/FLOP normalization and Adam or gradient-clipping baselines were not tested.

How to run: python3 bench_escape_controller.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Worked ✓

Agent confidence: 7/10

The escape-threshold controller significantly improved test MSE on the registered dynamics track with the shared rnn_small architecture. It beat tuned SGD on all 8 paired seeds, with delta_mean=-0.148157 and permutation p=0.0081. The trained-model mechanism signature was measured but not confirmed because observed rho was 0.0123–0.0232 rather than the target 0.70.

Baseline: Tuned SGD lr=0.01; full 8-seed test MSE mean=0.182376.
Idea: Adaptive controller lr=0.01; full 8-seed test MSE mean=0.034219; 8/8 wins; paired delta=-0.148157; permutation p=0.0081.

Benchmark result