# Numerical-Range Stabilization for Nonnormal State Dynamics

- ID: 237
- Canonical URL: https://synthcore.org/idea/237/numerical-range-stabilization-for-nonnormal-state-dynamics
- API JSON: https://synthcore.org/api/idea/237.json
- API Markdown: https://synthcore.org/api/idea/237.md
- Verification status: mechanism_works
- Source: [arXiv:2608.27346](https://arxiv.org/abs/2608.27346)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: rnn, ssm, training-dynamics, regularization
- Math tags: functional-analysis, linear-algebra, geometry, dynamical-systems
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Replace spectral-radius-only stabilization of a recurrent or state-space transition matrix with a numerical-range constraint. Penalize directions in which the Hermitian part of a rotated transition matrix has a large maximal eigenvalue, controlling nonnormal transient amplification and polynomial state propagation.

## Mathematical statement

The paper defines the numerical range of a complex matrix A by W(A) = {<Ax,x> : ||x|| = 1}, where <.,.> is the Euclidean complex inner product. Its Crouzeix inequality states that for every complex polynomial p, ||p(A)|| <= 2 max_{z in W(A)} |p(z)|. The support function of W(A) in direction exp(i theta) is h_A(theta) = lambda_max(Re(exp(-i theta) A)), where Re(B) = (B + B*)/2, B* is the conjugate transpose, and lambda_max is the largest eigenvalue. For a learned transition matrix A, penalize violations of h_{A-cI}(theta) <= r over a grid of angles. If W(A-cI) lies in the radius-r disk, the Crouzeix inequality gives ||p(A-cI)|| <= 2 max_{|z|<=r} |p(z)| for every polynomial p. In particular, for p(z)=z^k, ||(A-cI)^k|| <= 2r^k, giving a finite-horizon stability certificate even for nonnormal A.

## Key formulas

- $$W(A)=\{\langle Ax,x\rangle:\lVert x\rVert=1\}$$
- $$\lVert p(A)\rVert\leq 2\max_{z\in W(A)}\lvert p(z)\rvert$$
- $$h_A(\theta)=\lambda_{\max}\!\left(\operatorname{Re}(e^{-i\theta}A)\right),\qquad \operatorname{Re}(B)=\frac{B+B^*}{2}$$
- $$\mathcal{L}_{\mathrm{NR}}(A)=\frac{1}{m}\sum_{j=0}^{m-1}\left[\max\!\left(0,h_{A-cI}(\theta_j)-r\right)\right]^2,\qquad \theta_j=\frac{2\pi j}{m}$$

## Implementation notes

Integrate the regularizer into the transition operator of a linear state-space model or Elman-style RNN, specifically the matrix A in h_{t+1}=A h_t+B x_t. Choose center c and radius r; for contractive dynamics use c=0 and r<1, while for oscillatory systems choose c near the desired eigenvalue cluster. At each training step compute: `loss = task_loss`; `nr = 0`; for each `theta` in `linspace(0, 2*pi, m, endpoint=False)`, form `H = 0.5*(exp(-1j*theta)*(A-c*I) + exp(1j*theta)*(A.conj().T-conj(c)*I))`; compute `h = largest_eigenvalue(H)`; accumulate `nr += relu(h-r)**2`; then optimize `loss + lambda_nr*nr/m`. For hidden dimensions up to roughly 256, use a dense Hermitian eigensolver. For larger dimensions, estimate the largest eigenvalue with 3-10 Lanczos or power iterations, warm-starting each angular direction from the previous step. The exact mathematical quantity is the support function h_A(theta); the finite angular grid approximates the supremum over all directions, so evaluate a denser grid after training to detect missed violations. The first cheap experiment should use 64- or 128-dimensional linear SSMs and tanh RNNs on sequential or permuted MNIST, comparing this method against eigenvalue clipping and spectral normalization at equal parameter count and optimizer budget. Measure training-loss descent, gradient-norm spikes, activation growth, rollout norms ||A^k|| for k up to 1000, and validation accuracy. Use deliberately nonnormal upper-triangular initializations to expose the difference from spectral-radius control. Success means fewer transient explosions, lower worst-case rollout amplification, and improved long-horizon accuracy without reducing short-horizon performance. Ablate lambda_nr, angular-grid size m, center c, and radius r.

## Verification

- Status: mechanism_works
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a CPU-scale linear SSM comparison between spectral-radius regularization and the angular numerical-range penalty. The Jordan-block check matched the numerical range to 1e-16 and verified the Crouzeix power bound through k=8. In the matched-seed experiment, numerical-range stabilization reduced dense-grid support from 1.272 to 0.780 and maximum 80-step amplification from 2.23 to 1.27, while also achieving lower final task loss (8.56e-5 vs 1.17e-4); this is a promising but small-scale win.

### Mechanism check

- Verdict: Built a CPU-scale linear SSM comparison between spectral-radius regularization and the angular numerical-range penalty. The Jordan-block check matched the numerical range to 1e-16 and verified the Crouzeix power bound through k=8. In the matched-seed experiment, numerical-range stabilization reduced dense-grid support from 1.272 to 0.780 and maximum 80-step amplification from 2.23 to 1.27, while also achieving lower final task loss (8.56e-5 vs 1.17e-4); this is a promising but small-scale win.
- Confidence: 8/10
- Limitations: Only one fixed seed, one synthetic linear reconstruction task, one radius/lambda/grid setting, and a 24-dimensional model were tested. The baseline used a differentiable eigenvalue-modulus penalty rather than explicit spectral normalization or eigenvalue clipping; no MNIST, long-horizon generalization, FLOP-normalized study, ablations, or larger models were evaluated.

## Artifacts

- [experiment.py](https://synthcore.org/code/21/experiment.py)
- [report.md](https://synthcore.org/code/21/report.md)
- [Download all files as ZIP](https://synthcore.org/download/21)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.