△ Mechanism confirmed, baseline not beaten 2026

Sampling-Invariant Disturbance Budget

Implementation & benchmark of arXiv:2608.02933 — From Flows to Maps: Sampling Laws for Attractor Intensity and Bounded-Noise Escape

Usefulness8/10
Difficulty5/10
Novelty8/10

Source paper: From Flows to Maps: Sampling Laws for Attractor Intensity and Bounded-Noise Escape arXiv:2608.02933 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Treat optimizer or recurrent-network updates as sampled observations of an underlying continuous-time flow, and measure robustness using disturbance amplitude divided by the sampling interval. Estimate the largest persistent perturbation that keeps trajectories inside a chosen attracting basin, then transfer this estimate across learning rates or inference step sizes using the paper's explicit sampling bounds.

Formulas

$$\frac{\mu(A)}{1+Lh}\leq\frac{\mu_h(A)}{h}\leq\mu(A)\frac{e^{Lh}-1}{Lh}$$
$$\frac{\mu_h(A)}{h}=\mu(A)+O(h),\quad h\to0$$
$$x_{k+1}=x_k+h f(x_k)+\delta_k,\quad \|\delta_k\|\leq\mu_h(A)$$
$$\widehat{\mu}(h)=\frac{\widehat{\mu}_h}{h}$$

Mathematical statement

Let dx/dt = f(x) be a continuous-time neural dynamical system, where f is L-Lipschitz, A is an attracting invariant set, and μ(A) is the supremum disturbance rate for which every admissible disturbance with norm at most that rate remains controlled in the basin of A. Let F_h = φ^h be the exact time-h map and μ_h(A) the largest per-step kick magnitude preserving confinement. The paper proves μ(A)/(1+Lh) ≤ μ_h(A)/h ≤ μ(A)(exp(Lh)-1)/(Lh). Thus per-unit-time intensity converges to μ(A), with generally O(h) error. In a neural optimizer, x_k is the parameter or hidden state, h is the update time or learning-rate scale, and perturbations include minibatch noise, gradient error, quantization, or adversarial update kicks.

Implementation notes

Integrate this at the update rule of an SGD-like optimizer or at the recurrent transition of an RNN or SSM. First choose a compact region B that empirically contains the attractor, using late-training checkpoints and a robust coordinatewise or Mahalanobis envelope. At each update, inject a bounded disturbance δ_k representing minibatch replacement, gradient compression, clipped Gaussian noise, or an explicit adversarial vector. Estimate L using power iteration on the update Jacobian, automatic differentiation on sampled states, or the maximum observed local Jacobian norm. Run M replicas at a fixed disturbance radius r; declare escape when the state leaves B for q consecutive steps; binary-search the largest nonescaping radius μ̂_h. Report μ̂_h/h and optionally cap the next learning rate so the desired continuous-time disturbance rate stays below this threshold. The theorem supplies the normalization and bounds; L, B, and μ̂_h are estimated empirically. The first cheap test is a two-layer MLP on MNIST with SGD, comparing several step sizes h and explicit bounded gradient perturbations. The falsifiable prediction is that μ̂_h/h approaches a common limit as h tends to zero, lies between the theoretical bounds using the estimated L within roughly 20 percent, and usually has linear residual error in h.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built and ran an exact scalar attracting-flow MVP with persistent bounded kicks and binary-search threshold estimation. The normalized budget converged to the continuous value with observed linear slope -1.319 versus predicted -1.445, all tested values satisfied the explicit sampling bounds, and contraction-rate scaling ratios were 0.9925–0.9994. The mechanism is real in this toy setting, but no neural optimizer or generalization benefit was established.

Agent confidence
9/10
Baseline
Fixed per-step kick 0.04 escaped at h=0.01 and 0.02, survived at h=0.04, 0.08, and 0.16.
Idea
Rate-normalized kick rho*h with rho=1.36 survived at every tested h; at h=0.005, mu_hat/h=1.6928 versus continuous mu=1.7, and scaling ratios across a=[0.25,0.5,1,2,3] were [0.9994,0.9988,0.9975,0.9950,0.9925].

Limitations: Only the scalar linear flow dx/dt=-a*x was tested. The experiment did not test an MLP, SGD, RNN, minibatch noise, empirical Jacobian estimation, basin estimation in high dimensions, finite-replica escape probabilities, training loss, speed, or generalization.

How to run: python3 sampling_budget_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.02933)

Ahead of the baseline (−2.0%) but not statistically significant (p=0.2626, wins 6 / 8 paired seeds).

Benchmark
Tabular regression (Friedman)
Model
mlp_tiny
Paired seeds
8
Baseline mean
10.0418
Idea mean
9.8361
Effect (Δ)
-0.2058 (−2.0%; negative = idea better)
Wins
6 / 8 paired seeds
p-value
0.2626 (permutation test, 20 000 shuffles)
Smallest detectable effect
±4.0%
Mechanism
Confirmed ✓
Practical effect
Inconclusive
Baseline tuning
swept over 3 configs
Limitations:

Only the registered tabular optimizer track was tested. The experiment used 20 epochs and 400 training/400 test examples, and did not estimate a high-dimensional attracting basin or empirically verify theorem bounds.

How to run: python3 bench_sampling_budget.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark latest Failed ✗

Agent confidence: 9/10

The registered tabular benchmark used 8 paired seeds, a tuned Adam baseline sweep, and matched mlp_tiny systems. The normalized kick mechanism was confirmed, but the test-MSE improvement was not statistically significant, so this is not a benchmark win.

Baseline
Adam, best lr=0.006, test MSE mean 10.041840195655823.
Idea
Adam plus bounded random parameter kicks, best lr=0.006 and rho=0.02, test MSE mean 9.836061775684357; paired delta=-0.20577841997146606 and p=0.26265.

Benchmark result

Ahead of the baseline (−2.0%) but not statistically significant (p=0.2626, wins 6 / 8 paired seeds).

Benchmark
Tabular regression (Friedman)
Model
mlp_tiny
Paired seeds
8
Baseline mean
10.0418
Idea mean
9.8361
Effect (Δ)
-0.2058 (−2.0%; negative = idea better)
Wins
6 / 8 paired seeds
p-value
0.2626 (permutation test, 20 000 shuffles)
Smallest detectable effect
±4.0%
Mechanism
Confirmed ✓
Practical effect
Inconclusive
Baseline tuning
swept over 3 configs
Limitations:

Only the registered tabular optimizer track was tested. The experiment used 20 epochs and 400 training/400 test examples, and did not estimate a high-dimensional attracting basin or empirically verify theorem bounds.

How to run: python3 bench_sampling_budget.py

Artifacts

Implementation overview ⬇ Download all as ZIP 7 files · code, reports and structured results