Sampling-Invariant Disturbance Budget
Implementation & benchmark of arXiv:2608.02933 — From Flows to Maps: Sampling Laws for Attractor Intensity and Bounded-Noise Escape
Source paper: From Flows to Maps: Sampling Laws for Attractor Intensity and Bounded-Noise Escape arXiv:2608.02933 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Treat optimizer or recurrent-network updates as sampled observations of an underlying continuous-time flow, and measure robustness using disturbance amplitude divided by the sampling interval. Estimate the largest persistent perturbation that keeps trajectories inside a chosen attracting basin, then transfer this estimate across learning rates or inference step sizes using the paper's explicit sampling bounds.
Formulas
Mathematical statement
Let dx/dt = f(x) be a continuous-time neural dynamical system, where f is L-Lipschitz, A is an attracting invariant set, and μ(A) is the supremum disturbance rate for which every admissible disturbance with norm at most that rate remains controlled in the basin of A. Let F_h = φ^h be the exact time-h map and μ_h(A) the largest per-step kick magnitude preserving confinement. The paper proves μ(A)/(1+Lh) ≤ μ_h(A)/h ≤ μ(A)(exp(Lh)-1)/(Lh). Thus per-unit-time intensity converges to μ(A), with generally O(h) error. In a neural optimizer, x_k is the parameter or hidden state, h is the update time or learning-rate scale, and perturbations include minibatch noise, gradient error, quantization, or adversarial update kicks.
Implementation notes
Integrate this at the update rule of an SGD-like optimizer or at the recurrent transition of an RNN or SSM. First choose a compact region B that empirically contains the attractor, using late-training checkpoints and a robust coordinatewise or Mahalanobis envelope. At each update, inject a bounded disturbance δ_k representing minibatch replacement, gradient compression, clipped Gaussian noise, or an explicit adversarial vector. Estimate L using power iteration on the update Jacobian, automatic differentiation on sampled states, or the maximum observed local Jacobian norm. Run M replicas at a fixed disturbance radius r; declare escape when the state leaves B for q consecutive steps; binary-search the largest nonescaping radius μ̂_h. Report μ̂_h/h and optionally cap the next learning rate so the desired continuous-time disturbance rate stays below this threshold. The theorem supplies the normalization and bounds; L, B, and μ̂_h are estimated empirically. The first cheap test is a two-layer MLP on MNIST with SGD, comparing several step sizes h and explicit bounded gradient perturbations. The falsifiable prediction is that μ̂_h/h approaches a common limit as h tends to zero, lies between the theoretical bounds using the estimated L within roughly 20 percent, and usually has linear residual error in h.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Confirmed ✓
Stage 2 · Practical benchmark result: Inconclusive
Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built and ran an exact scalar attracting-flow MVP with persistent bounded kicks and binary-search threshold estimation. The normalized budget converged to the continuous value with observed linear slope -1.319 versus predicted -1.445, all tested values satisfied the explicit sampling bounds, and contraction-rate scaling ratios were 0.9925–0.9994. The mechanism is real in this toy setting, but no neural optimizer or generalization benefit was established.
- Agent confidence
- 9/10
- Baseline
- Fixed per-step kick 0.04 escaped at h=0.01 and 0.02, survived at h=0.04, 0.08, and 0.16.
- Idea
- Rate-normalized kick rho*h with rho=1.36 survived at every tested h; at h=0.005, mu_hat/h=1.6928 versus continuous mu=1.7, and scaling ratios across a=[0.25,0.5,1,2,3] were [0.9994,0.9988,0.9975,0.9950,0.9925].
Limitations: Only the scalar linear flow dx/dt=-a*x was tested. The experiment did not test an MLP, SGD, RNN, minibatch noise, empirical Jacobian estimation, basin estimation in high dimensions, finite-replica escape probabilities, training loss, speed, or generalization.
How to run: python3 sampling_budget_experiment.py
Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.02933)
Ahead of the baseline (−2.0%) but not statistically significant (p=0.2626, wins 6 / 8 paired seeds).
- Benchmark
- Tabular regression (Friedman)
- Model
- mlp_tiny
- Paired seeds
- 8
- Baseline mean
- 10.0418
- Idea mean
- 9.8361
- Effect (Δ)
- -0.2058 (−2.0%; negative = idea better)
- Wins
- 6 / 8 paired seeds
- p-value
- 0.2626 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±4.0%
- Mechanism
- Confirmed ✓
- Practical effect
- Inconclusive
- Baseline tuning
- swept over 3 configs
Only the registered tabular optimizer track was tested. The experiment used 20 epochs and 400 training/400 test examples, and did not estimate a high-dimensional attracting basin or empirically verify theorem bounds.
How to run: python3 bench_sampling_budget.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Stage 2 — Benchmark latest Failed ✗
Agent confidence: 9/10
The registered tabular benchmark used 8 paired seeds, a tuned Adam baseline sweep, and matched mlp_tiny systems. The normalized kick mechanism was confirmed, but the test-MSE improvement was not statistically significant, so this is not a benchmark win.
- Baseline
- Adam, best lr=0.006, test MSE mean 10.041840195655823.
- Idea
- Adam plus bounded random parameter kicks, best lr=0.006 and rho=0.02, test MSE mean 9.836061775684357; paired delta=-0.20577841997146606 and p=0.26265.
Benchmark result
Ahead of the baseline (−2.0%) but not statistically significant (p=0.2626, wins 6 / 8 paired seeds).
- Benchmark
- Tabular regression (Friedman)
- Model
- mlp_tiny
- Paired seeds
- 8
- Baseline mean
- 10.0418
- Idea mean
- 9.8361
- Effect (Δ)
- -0.2058 (−2.0%; negative = idea better)
- Wins
- 6 / 8 paired seeds
- p-value
- 0.2626 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±4.0%
- Mechanism
- Confirmed ✓
- Practical effect
- Inconclusive
- Baseline tuning
- swept over 3 configs
Only the registered tabular optimizer track was tested. The experiment used 20 epochs and 400 training/400 test examples, and did not estimate a high-dimensional attracting basin or empirically verify theorem bounds.
How to run: python3 bench_sampling_budget.py
Artifacts
- REPORT.md 2.4 KB View
- bench_report.json 4.9 KB View Raw
- bench_sampling_budget.py 5.6 KB View Raw
- report.md 1.4 KB View
- report_bench_2026-08-31T173138.md 3.3 KB View
- sampling_budget_experiment.py 4.6 KB View Raw
- sampling_budget_results.json 5.1 KB View Raw