# Global Basin Continuation for Neural Dynamics

- ID: 632
- Canonical URL: https://synthcore.org/idea/632/global-basin-continuation-for-neural-dynamics
- API JSON: https://synthcore.org/api/idea/632.json
- API Markdown: https://synthcore.org/api/idea/632.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.09332](https://arxiv.org/abs/2607.09332)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: rnn, ssm, training-dynamics, optimizer
- Math tags: dynamical-systems, bifurcations, probability, statistics, control-theory
- Ratings: usefulness 7/10; difficulty 6/10; novelty 8/10

## Idea description

Treat the hidden-state evolution of an RNN or state-space model as a parameterized dynamical system and globally continue its attractors over a grid of inputs, perturbation amplitudes, and training checkpoints. Penalize or stop training when the task-relevant attractor loses basin mass, rather than relying only on local Jacobian eigenvalues at one nominal trajectory.

## Mathematical statement

Let z_{t+1}=F_{theta,u}(z_t) be a discrete-time neural dynamical system with parameters theta and fixed input or operating condition u. Let A_i(theta,u) denote an attractor and let X be a perturbation region with sampling measure mu. Its basin fraction is B_i(theta,u)=mu({z_0 in X: F_{theta,u}^t(z_0) converges to A_i}) divided by mu(X). If an observed rollout reaches attractor A_0, Bayesian conditioning gives P(theta | A=A_0)=P(A=A_0 | theta)P(theta)/P(A=A_0), with likelihood estimated by B_0(theta,u). The transferable mechanism is that a finite-perturbation stability loss based on 1 minus B_0 detects basin collapse and attractor switching that local linear stability can miss.

## Key formulas

- $$B_i(\theta,u)=\frac{\mu\left(\left\{z_0\in\mathcal{X}:\lim_{t\to\infty}F_{\theta,u}^{t}(z_0)\in A_i(\theta,u)\right\}\right)}{\mu(\mathcal{X})}$$
- $$\widehat{B}_i(\theta,u)=\frac{1}{N}\sum_{k=1}^{N}\mathbf{1}\left[\operatorname{Classify}\left(F_{\theta,u}^{T}(z_0^{(k)})\right)=A_i\right],\qquad z_0^{(k)}\sim\mu$$
- $$\mathrm{P}(\theta\mid A=A_0)=\frac{\mathrm{P}(A=A_0\mid\theta)\,\mathrm{P}(\theta)}{\mathrm{P}(A=A_0)},\qquad \mathrm{P}(A=A_0\mid\theta)\approx\widehat{B}_0(\theta,u)$$
- $$\mathcal{L}_{\mathrm{total}}(\theta)=\mathcal{L}_{\mathrm{task}}(\theta)+\lambda\,\max\left(0,B_{\min}-\widehat{B}_0(\theta,u)\right)^2$$

## Implementation notes

1. Integration point: use a tanh GRU, vanilla RNN, or diagonal state-space model whose hidden state is iterated for a fixed input context until it reaches a recurrent regime. Select a task-relevant attractor by continuing the unperturbed trajectory. For chaotic or nonconvergent systems, replace point-attractor classification by clustering terminal windows or recurrence statistics. At each training checkpoint, perturb the hidden state with N samples from a prescribed ball or empirical hidden-state distribution.

2. Pseudocode: save the clean hidden state z_star; sample z_0^(k)=z_star+epsilon xi_k, where xi_k are normalized random vectors; iterate z_{t+1}=F_{theta,u}(z_t) for T steps; cluster terminal states or compare them with the clean attractor; estimate B_hat_0; add lambda times max(0,B_min-B_hat_0)^2 to the task loss, or reject the optimizer step if the basin fraction falls by more than delta. Repeat for a coarse grid of operating conditions u and perturbation amplitudes epsilon.

3. Computed versus estimated: the update map F_{theta,u} and task loss are computed exactly by the network. Attractors, basin labels, and B_i are estimated empirically from finite rollouts; use Wilson confidence intervals and increase N only near a suspected transition. Warm-start perturbation trajectories from neighboring parameter-grid points, while retaining all discovered attractor clusters instead of following only the nearest one.

4. First cheap experiment: train a two-layer tanh RNN on sequential MNIST or the adding-two-numbers task, comparing Adam, Adam plus spectral normalization, and Adam plus the basin penalty. At checkpoints, measure B_hat_0(epsilon) for epsilon from 10^-3 to 1, the number of hidden-state attractor clusters, long-horizon task error, and the local Jacobian spectral radius. The falsifiable prediction is that long-horizon failures begin when B_hat_0 drops sharply, often before the nominal Jacobian crosses spectral radius one. Across seeds, the perturbation threshold epsilon_c satisfying B_hat_0(epsilon_c)=0.5 should predict rollout failure within 20 percent. The basin penalty should shift epsilon_c upward and suppress abrupt attractor switching, even if the local Jacobian margin changes little.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: inconclusive
- Verdict: Built a scalar neural-dynamics global-basin continuation MVP with exact basin calculations, Monte Carlo estimation, local Jacobian comparison, and a basin-penalized optimization toy experiment. The predicted onset epsilon_c=sqrt(a) and B=0.75 crossing epsilon_c=2sqrt(a) matched sweeps within about 1%, while Monte Carlo basin estimates had mean absolute error 0.00077. Finite basin switching occurred despite local spectral radius remaining below one, and the penalty increased final basin fraction from 0.60 to 0.778, though at higher task error; this confirms the mechanism in the toy setting, not yet a neural-task win.

### Mechanism check

- Verdict: Built a scalar neural-dynamics global-basin continuation MVP with exact basin calculations, Monte Carlo estimation, local Jacobian comparison, and a basin-penalized optimization toy experiment. The predicted onset epsilon_c=sqrt(a) and B=0.75 crossing epsilon_c=2sqrt(a) matched sweeps within about 1%, while Monte Carlo basin estimates had mean absolute error 0.00077. Finite basin switching occurred despite local spectral radius remaining below one, and the penalty increased final basin fraction from 0.60 to 0.778, though at higher task error; this confirms the mechanism in the toy setting, not yet a neural-task win.
- Confidence: 9/10
- Limitations: This is a one-dimensional polynomial dynamical-system proxy rather than a tanh RNN or sequence-learning task. The basin estimate uses a known analytic basin boundary, optimization uses the exact basin fraction rather than noisy finite-sample estimates, and no multi-seed FLOP-matched neural baseline, checkpoint continuation, Wilson intervals, or high-dimensional attractor clustering was tested.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.34565
- Paired wins: 5/8
- Benchmark verdict: no significant win

## Artifacts

- [basin_continuation_experiment.py](https://synthcore.org/code/284/basin_continuation_experiment.py)
- [basin_results.json](https://synthcore.org/code/284/basin_results.json)
- [bench_report.json](https://synthcore.org/code/284/bench_report.json)
- [report.md](https://synthcore.org/code/284/report.md)
- [report_bench_2026-08-31T001856.md](https://synthcore.org/code/284/report_bench_2026-08-31T001856.md)
- [stage2_bench.py](https://synthcore.org/code/284/stage2_bench.py)
- [stage2_stdout.json](https://synthcore.org/code/284/stage2_stdout.json)
- [Download all files as ZIP](https://synthcore.org/download/284)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.