# Noise-Threshold Basin Merging for Recurrent Memory

- ID: 1470
- Canonical URL: https://synthcore.org/idea/1470/noise-threshold-basin-merging-for-recurrent-memory
- API JSON: https://synthcore.org/api/idea/1470.json
- API Markdown: https://synthcore.org/api/idea/1470.md
- Verification status: unverified
- Source: [arXiv:2607.26963](https://arxiv.org/abs/2607.26963)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: rnn, ssm, regularization, training-dynamics
- Math tags: dynamical-systems, bifurcations, statistics, linear-algebra
- Ratings: usefulness 6/10; difficulty 6/10; novelty 8/10

## Idea description

Use attractor separation and noise-induced basin coalescence as a robustness test for recurrent networks with multiple learned memories or modes. Estimate the smallest perturbation amplitude at which initially distinct hidden-state attractors become geometrically indistinguishable, then train or operate below that threshold with a safety margin.

## Mathematical statement

Let F_{theta,xi} be a recurrent state map with independent perturbations xi_t of amplitude sigma, and let A_i(sigma) be the empirical attractor reached from initialization cluster i. Define d_{ij}(sigma) as the distance between attractor centroids and r_i(sigma) as the within-attractor radius. Basin merging is declared when d_{ij}(sigma) is no larger than kappa times the sum of the two radii, where kappa is fixed before testing. The critical noise level is the smallest sigma satisfying this condition for any pair. FTLE histograms provide an auxiliary diagnostic because separated stable regions tend to have negative finite-time exponents, whereas chaotic or basin-boundary regions have a larger positive-FTLE fraction.

## Key formulas

- $$h_{t+1}=F_{\theta+\xi_t}(h_t,u_t),\qquad \xi_t\stackrel{\mathrm{i.i.d.}}{\sim}\mathcal{D}_{\sigma}.$$
- $$\sigma_c=\inf\left\{\sigma:\ d_{ij}(\sigma)\leq\kappa\left[r_i(\sigma)+r_j(\sigma)\right]\text{ for some }i\neq j\right\}.$$
- $$\lambda_T(h_0)=\frac{1}{T}\log\left\|\prod_{t=0}^{T-1}\frac{\partial F_{\theta+\xi_t}}{\partial h}(h_t,u_t)\right\|.$$
- $$\mathcal{L}_{\mathrm{margin}}=\gamma\left[\sigma_{\mathrm{safe}}-\widehat{\sigma}_c\right]_+^2,\qquad\sigma_{\mathrm{safe}}=(1-m)\widehat{\sigma}_c.$$

## Implementation notes

Integration point: use a recurrent classifier, associative-memory RNN, or SSM whose hidden state should preserve multiple modes. Add parameter noise, dropout masks, or input perturbations independently at each recurrent step. Pseudocode: choose several prototype initial states or examples for each intended mode; for a logarithmic grid of sigma values, run R perturbed trajectories per prototype for T steps; cluster terminal hidden states; compute cluster radii r_i and centroid distances d_{ij}; simultaneously estimate FTLEs with Jacobian-vector products. Declare merging when the distance criterion holds in at least 80 percent of trials. Estimate sigma_c by bisection between the largest nonmerged and smallest merged noise level, and constrain the deployed noise or augmentation to (1-m)sigma_c. The paper supplies the finite-time Jacobian diagnostic, geometric attractor separation, and critical-noise bracketing strategy. Cluster assignments, radii, distances, and the threshold are estimated empirically, preferably with bootstrap confidence intervals. First experiment: train a two-mode tanh RNN on a synthetic sequence task where the mode is encoded by the initial hidden state, then measure noisy recall while sweeping Gaussian weight noise. Compare ordinary validation-based noise selection with threshold-aware selection. The prediction is a kink or rapid increase in mode-confusion rate near sigma_c, coincident with d_{ij}/(r_i+r_j) crossing kappa and a rise in the positive-FTLE fraction. The estimated threshold should match the independently measured 50 percent confusion point within 20 percent.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.