# Black-Box Neural Interconnection Stability Margin

- ID: 737
- Canonical URL: https://synthcore.org/idea/737/black-box-neural-interconnection-stability-margin
- API JSON: https://synthcore.org/api/idea/737.json
- API Markdown: https://synthcore.org/api/idea/737.md
- Verification status: beats_baseline
- Source: [arXiv:2607.11243](https://arxiv.org/abs/2607.11243)
- Category: dynamics
- Solves: stability, accuracy
- ML areas: rnn, ssm, training-dynamics, regularization
- Math tags: control-theory, dynamical-systems, linear-algebra, spectral-theory
- Ratings: usefulness 7/10; difficulty 6/10; novelty 7/10

## Idea description

Treat recurrent or state-space network blocks as measured dynamical components and analyze their closed-loop interaction through frequency-domain gain, without requiring exact internal state-space equations. Estimate each block's local transfer matrix from perturbation-response experiments, assemble the block interconnection, and regularize training whenever the interaction approaches a small-gain or singularity boundary.

## Mathematical statement

For module j, let u_j(t) be an input perturbation and y_j(t) its output perturbation. Estimate the local frequency response G_j(i omega) by dividing the Fourier transform of the output by the Fourier transform of the input across multiple probe directions. Let G(i omega) be the block-diagonal matrix of all component responses and C(i omega) encode cross-block wiring, residual paths, feedback, or normalization couplings. The closed-loop perturbation equation is y = G(u + Cy), with return operator L = GC. A conservative sufficient stability condition is gamma = max over sampled frequencies of the largest singular value of L less than 1. The margin m = 1 - gamma is positive in the contraction regime and becomes nonpositive at a predicted amplification boundary.

## Key formulas

- $$\hat{G}_j(i\omega)=\frac{\widehat{Y_j(i\omega)}}{\widehat{U_j(i\omega)}},\qquad \widehat{Y_j(i\omega)}=\mathcal{F}[y_j](\omega),\quad \widehat{U_j(i\omega)}=\mathcal{F}[u_j](\omega)$$
- $$y=G(i\omega)(u+Cy),\qquad y=(I-L(i\omega))^{-1}G(i\omega)u,\qquad L(i\omega)=G(i\omega)C(i\omega)$$
- $$\gamma=\max_{\omega\in\Omega}\sigma_{\max}(L(i\omega)),\qquad m=1-\gamma,\qquad \mathcal{L}_{\mathrm{stab}}=\lambda\left[\max(0,\gamma-\gamma_0)\right]^2$$
- $$\|\delta y_k\|\lesssim \gamma^k\|\delta y_0\|\quad\text{when }\gamma<1$$

## Implementation notes

Integrate this at the block level of a residual RNN, SSM, implicit layer, or recurrent world model. Select operating points from normal training batches and periodically freeze the weights for a probe pass. For every module, inject a small zero-mean perturbation at its input using 8 to 16 sinusoidal frequencies or one broadband sequence, and record the output perturbation. Use FFT ratios and several probe directions to estimate every input-output entry of the local response matrix. Assemble G as a block-diagonal matrix. Construct C from the known computational graph, or estimate it by perturbing one block while observing the immediate input of the other blocks. At each sampled frequency compute the largest singular value of L = GC, then add the stated hinge penalty when gamma exceeds gamma_0. The component-response estimation is inherited from the paper; probe amplitude, frequency grid, and penalty threshold are empirical choices. First test a two-block GRU or linear SSM on sequential MNIST and the copy-memory task, comparing ordinary training with the margin penalty. Sweep a residual coupling scalar a and plot gamma(a) against hidden-state perturbation growth. The falsifiable prediction is a transition near gamma = 1: systems with gamma below 0.9 should have bounded perturbation amplification, while systems with gamma above 1 should show increasing long-horizon error. The measured transition should agree with the singular-value boundary within 20 percent.

## Verification

- Status: beats_baseline
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: helps
- Verdict: Built a black-box FFT transfer estimator, closed-loop singular-value margin calculation, perturbation-growth sweep, and hinge-penalty training prototype in verify_stability.py. The three toy predictions manifested: gamma(a) slope was 0.9000 versus predicted 0.9000, the observed growth boundary was a=1.12 versus predicted 1.1111, and resolvent scaling matched 1/(1-gamma) with 3.9e-16 relative error. The penalty did not improve the tiny recurrent fitting run because both baseline and penalty converged to the same stable coefficient and loss; this verifies the mechanism, not a demonstrated training win.

### Mechanism check

- Verdict: Built a black-box FFT transfer estimator, closed-loop singular-value margin calculation, perturbation-growth sweep, and hinge-penalty training prototype in verify_stability.py. The three toy predictions manifested: gamma(a) slope was 0.9000 versus predicted 0.9000, the observed growth boundary was a=1.12 versus predicted 1.1111, and resolvent scaling matched 1/(1-gamma) with 3.9e-16 relative error. The penalty did not improve the tiny recurrent fitting run because both baseline and penalty converged to the same stable coefficient and loss; this verifies the mechanism, not a demonstrated training win.
- Confidence: 8/10
- Limitations: Only a symmetric two-block static linear toy was tested, deliberately making the singular-value boundary exact. No GRU/SSM, sequential MNIST, copy-memory task, frequency-dependent dynamics, noisy C estimation, GPU execution, or meaningful large-scale training comparison was tested; the regularizer showed no benefit in the tiny identification experiment.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.02945
- Paired wins: 6/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/309/bench_report.json)
- [report.md](https://synthcore.org/code/309/report.md)
- [report_bench_2026-08-31T005327.md](https://synthcore.org/code/309/report_bench_2026-08-31T005327.md)
- [results.json](https://synthcore.org/code/309/results.json)
- [stage2_stability_bench.py](https://synthcore.org/code/309/stage2_stability_bench.py)
- [verify_stability.py](https://synthcore.org/code/309/verify_stability.py)
- [Download all files as ZIP](https://synthcore.org/download/309)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.