# Eigenvalue-Sensitivity Stability Margin

- ID: 1552
- Canonical URL: https://synthcore.org/idea/1552/eigenvalue-sensitivity-stability-margin
- API JSON: https://synthcore.org/api/idea/1552.json
- API Markdown: https://synthcore.org/api/idea/1552.md
- Verification status: mechanism_failed
- Source: [arXiv:2607.28764](https://arxiv.org/abs/2607.28764)
- Category: dynamics
- Solves: stability, accuracy
- ML areas: rnn, ssm, optimizer, training-dynamics, regularization
- Math tags: dynamical-systems, control-theory, spectral-theory, linear-algebra, numerical-analysis
- Ratings: usefulness 8/10; difficulty 6/10; novelty 7/10

## Idea description

Add a sensitivity-aware stability monitor and regularizer to an RNN, neural state-space model, or linearized sequence model. Instead of evaluating the model at many perturbed inputs or parameter settings, estimate how each perturbation changes the dominant eigenvalues of the local hidden-state Jacobian, then penalize perturbations predicted to push eigenvalues toward the unit circle. This should improve long-horizon behavior while identifying a quantitative perturbation radius at which instability is expected.

## Mathematical statement

Let the neural state update be h_(t+1) = f_theta(h_t, u_t), and let A(h_t,u_t) = partial f_theta / partial h be its local state-transition Jacobian. Let lambda_i be a simple eigenvalue of A, with right eigenvector v_i and left eigenvector w_i satisfying A v_i = lambda_i v_i and w_i^H A = lambda_i w_i^H. For a scalar perturbation p, the first-order eigenvalue sensitivity is d lambda_i / d p = [w_i^H (partial A / partial p) v_i] / [w_i^H v_i]. If the operating point z is constrained by g(z,p) = 0, implicit differentiation gives partial z / partial p = -J^(-1)(partial g / partial p), where J = partial g / partial z. Define the discrete-time spectral margin m = 1 - rho(A), where rho(A) is the largest eigenvalue modulus. Along perturbation direction q, estimate the first-order crossing radius delta_hat_c as the smallest ratio between the distance 1 - abs(lambda_i) and the radial eigenvalue drift induced by q. Penalize small margins and large sensitivities.

## Key formulas

- $$\frac{\partial z}{\partial p}=-J^{-1}\frac{\partial g}{\partial p},\qquad J=\frac{\partial g}{\partial z}.$$
- $$\frac{d\lambda_i}{dp}=\frac{w_i^{\mathsf H}\left(\frac{\partial A}{\partial p}\right)v_i}{w_i^{\mathsf H}v_i},\qquad Av_i=\lambda_i v_i,\quad w_i^{\mathsf H}A=\lambda_i w_i^{\mathsf H}.$$
- $$m=1-\rho(A),\qquad \rho(A)=\max_i|\lambda_i|,$$
- $$\hat{\delta}_c=\min_i\frac{1-|\lambda_i|}{\left|\operatorname{Re}\left(e^{-\mathrm{i}\arg\lambda_i}\frac{d\lambda_i}{dp}q\right)\right|}.$$

## Implementation notes

1. Exact integration point: use a recurrent cell, diagonal-plus-low-rank state-space model, or neural state-space predictor. During training, select representative hidden states from each minibatch and form the local Jacobian A = partial f_theta / partial h using automatic differentiation, Jacobian-vector products, or an explicit matrix for small hidden dimensions. Treat p as a low-dimensional perturbation such as input mean, input variance, recurrent gain, normalization scale, or a connectivity-mask amplitude. 2. Pseudocode: compute A; obtain the k eigenvalues closest to the unit circle and their left and right eigenvectors; estimate partial A / partial p using finite differences A(p plus epsilon) and A(p minus epsilon), or second-order autodiff; calculate each d lambda_i / d p with the eigenvector formula; compute m and delta_hat_c; add a penalty for m below a target m_0 and optionally a squared sensitivity penalty; backpropagate the task loss plus this penalty. 3. Computed versus estimated: eigenvalues and eigenvectors are computed locally, while derivatives of A may initially be estimated with two additional Jacobian evaluations. The crossing radius is a first-order prediction and must be checked by explicitly recomputing eigenvalues after perturbation. 4. First cheap experiment: compare vanilla tanh RNNs and sensitivity-regularized RNNs on Copy Memory and adding problems with identical hidden size and optimizer. Sweep recurrent-gain and input-statistics perturbations after training. The falsifiable signature is that the measured dominant-eigenvalue crossing occurs within 20 percent of delta_hat_c for sufficiently small perturbations, while the regularized model has a larger median predicted crossing radius and delayed long-horizon error explosion. Also plot the dominant eigenvalue modulus against perturbation size; the initial slope should match the sensitivity estimate.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built an eigenvalue-sensitivity module with exact local eigensensitivity, finite-difference validation, crossing-radius sweeps, and a CUDA-capable tanh-RNN regularization comparison. The core derivative formula worked extremely well, and the median predicted crossing radius was within 7.7% of the measured crossing, satisfying the stated 20% signal. However, inverse-gain and linear-margin scaling were not confirmed (CV 18.97% and 9.08%), and the RNN comparison was mixed: the idea increased predicted radius from 0.155 to 0.194 and reduced nominal/noise-2 output error, but worsened the gain-perturbed output error and maximum hidden norm, so there is no reliable demonstrated stability win.

### Mechanism check

- Verdict: Built an eigenvalue-sensitivity module with exact local eigensensitivity, finite-difference validation, crossing-radius sweeps, and a CUDA-capable tanh-RNN regularization comparison. The core derivative formula worked extremely well, and the median predicted crossing radius was within 7.7% of the measured crossing, satisfying the stated 20% signal. However, inverse-gain and linear-margin scaling were not confirmed (CV 18.97% and 9.08%), and the RNN comparison was mixed: the idea increased predicted radius from 0.155 to 0.194 and reduced nominal/noise-2 output error, but worsened the gain-perturbed output error and maximum hidden norm, so there is no reliable demonstrated stability win.
- Confidence: 8/10
- Limitations: Only one small tanh-RNN task and one random seed were tested; no Copy Memory benchmark, FLOP-normalized study, repeated-seed statistics, explicit operating-point perturbation distribution, or long-horizon task accuracy metric was evaluated. The training regularizer uses differentiable spectral-radius evaluations at two recurrent gains rather than a full eigenvector-sensitivity penalty for every perturbation direction.

## Artifacts

- [experiment.py](https://synthcore.org/code/391/experiment.py)
- [report.md](https://synthcore.org/code/391/report.md)
- [results.json](https://synthcore.org/code/391/results.json)
- [Download all files as ZIP](https://synthcore.org/download/391)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.