# Sampling-Invariant Disturbance Budget

- ID: 1738
- Canonical URL: https://synthcore.org/idea/1738/sampling-invariant-disturbance-budget
- API JSON: https://synthcore.org/api/idea/1738.json
- API Markdown: https://synthcore.org/api/idea/1738.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.02933](https://arxiv.org/abs/2608.02933)
- Category: dynamics
- Solves: stability, generalization
- ML areas: optimizer, training-dynamics, rnn
- Math tags: dynamical-systems, control-theory, numerical-analysis
- Ratings: usefulness 8/10; difficulty 5/10; novelty 8/10

## Idea description

Treat optimizer or recurrent-network updates as sampled observations of an underlying continuous-time flow, and measure robustness using disturbance amplitude divided by the sampling interval. Estimate the largest persistent perturbation that keeps trajectories inside a chosen attracting basin, then transfer this estimate across learning rates or inference step sizes using the paper's explicit sampling bounds.

## Mathematical statement

Let dx/dt = f(x) be a continuous-time neural dynamical system, where f is L-Lipschitz, A is an attracting invariant set, and μ(A) is the supremum disturbance rate for which every admissible disturbance with norm at most that rate remains controlled in the basin of A. Let F_h = φ^h be the exact time-h map and μ_h(A) the largest per-step kick magnitude preserving confinement. The paper proves μ(A)/(1+Lh) ≤ μ_h(A)/h ≤ μ(A)(exp(Lh)-1)/(Lh). Thus per-unit-time intensity converges to μ(A), with generally O(h) error. In a neural optimizer, x_k is the parameter or hidden state, h is the update time or learning-rate scale, and perturbations include minibatch noise, gradient error, quantization, or adversarial update kicks.

## Key formulas

- $$\frac{\mu(A)}{1+Lh}\leq\frac{\mu_h(A)}{h}\leq\mu(A)\frac{e^{Lh}-1}{Lh}$$
- $$\frac{\mu_h(A)}{h}=\mu(A)+O(h),\quad h\to0$$
- $$x_{k+1}=x_k+h f(x_k)+\delta_k,\quad \|\delta_k\|\leq\mu_h(A)$$
- $$\widehat{\mu}(h)=\frac{\widehat{\mu}_h}{h}$$

## Implementation notes

Integrate this at the update rule of an SGD-like optimizer or at the recurrent transition of an RNN or SSM. First choose a compact region B that empirically contains the attractor, using late-training checkpoints and a robust coordinatewise or Mahalanobis envelope. At each update, inject a bounded disturbance δ_k representing minibatch replacement, gradient compression, clipped Gaussian noise, or an explicit adversarial vector. Estimate L using power iteration on the update Jacobian, automatic differentiation on sampled states, or the maximum observed local Jacobian norm. Run M replicas at a fixed disturbance radius r; declare escape when the state leaves B for q consecutive steps; binary-search the largest nonescaping radius μ̂_h. Report μ̂_h/h and optionally cap the next learning rate so the desired continuous-time disturbance rate stays below this threshold. The theorem supplies the normalization and bounds; L, B, and μ̂_h are estimated empirically. The first cheap test is a two-layer MLP on MNIST with SGD, comparing several step sizes h and explicit bounded gradient perturbations. The falsifiable prediction is that μ̂_h/h approaches a common limit as h tends to zero, lies between the theoretical bounds using the estimated L within roughly 20 percent, and usually has linear residual error in h.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: inconclusive
- Verdict: Built and ran an exact scalar attracting-flow MVP with persistent bounded kicks and binary-search threshold estimation. The normalized budget converged to the continuous value with observed linear slope -1.319 versus predicted -1.445, all tested values satisfied the explicit sampling bounds, and contraction-rate scaling ratios were 0.9925–0.9994. The mechanism is real in this toy setting, but no neural optimizer or generalization benefit was established.

### Mechanism check

- Verdict: Built and ran an exact scalar attracting-flow MVP with persistent bounded kicks and binary-search threshold estimation. The normalized budget converged to the continuous value with observed linear slope -1.319 versus predicted -1.445, all tested values satisfied the explicit sampling bounds, and contraction-rate scaling ratios were 0.9925–0.9994. The mechanism is real in this toy setting, but no neural optimizer or generalization benefit was established.
- Confidence: 9/10
- Limitations: Only the scalar linear flow dx/dt=-a*x was tested. The experiment did not test an MLP, SGD, RNN, minibatch noise, empirical Jacobian estimation, basin estimation in high dimensions, finite-replica escape probabilities, training loss, speed, or generalization.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.26265
- Paired wins: 6/8
- Benchmark verdict: no significant win

## Artifacts

- [REPORT.md](https://synthcore.org/code/550/REPORT.md)
- [bench_report.json](https://synthcore.org/code/550/bench_report.json)
- [bench_sampling_budget.py](https://synthcore.org/code/550/bench_sampling_budget.py)
- [report.md](https://synthcore.org/code/550/report.md)
- [report_bench_2026-08-31T173138.md](https://synthcore.org/code/550/report_bench_2026-08-31T173138.md)
- [sampling_budget_experiment.py](https://synthcore.org/code/550/sampling_budget_experiment.py)
- [sampling_budget_results.json](https://synthcore.org/code/550/sampling_budget_results.json)
- [Download all files as ZIP](https://synthcore.org/download/550)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.