# Numerical-range regularization for nonnormal state dynamics

- ID: 1760
- Canonical URL: https://synthcore.org/idea/1760/numerical-range-regularization-for-nonnormal-state-dynamics
- API JSON: https://synthcore.org/api/idea/1760.json
- API Markdown: https://synthcore.org/api/idea/1760.md
- Verification status: unverified
- Source: [arXiv:2608.03841](https://arxiv.org/abs/2608.03841)
- Category: dynamics
- Solves: stability, generalization
- ML areas: ssm, rnn, training-dynamics, regularization
- Math tags: functional-analysis, linear-algebra, numerical-analysis, dynamical-systems
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Constrain the numerical range of a learned recurrent or state-space transition matrix instead of constraining only its eigenvalues or singular norm. The resulting Crouzeix certificate controls every polynomial time filter, including multi-step powers and residual propagation, and is designed to suppress transient amplification caused by nonnormality.

## Mathematical statement

The paper's central result states that for any linear operator A on a finite-dimensional complex Hilbert space and any complex polynomial p, ||p(A)|| is at most 2 times the maximum of |p(z)| over the numerical range. Here ||.|| is the Euclidean operator norm and W(A)={x*Ax: x is a unit vector} is the numerical range. For p(z)=z^k, the result gives ||A^k||_2 <= 2 w(A)^k, where k is the rollout length and w(A)=max_{||x||_2=1}|x*Ax| is the numerical radius. The numerical-range support function is h_A(theta)=lambda_max(Re(exp(-i theta)A)), where Re(B)=(B+B*)/2 and theta is an angle. Sampling angles therefore gives an estimate of the numerical radius. The proposed neural-network surrogate penalizes [w_hat(A)-rho]_+^2, or directly penalizes predicted multi-step bounds. The theorem supplies the factor-2 bound for every polynomial; the angle grid, eigenvalue approximation, radius target rho, and rollout horizons are engineering choices.

## Key formulas

- $$\|p(A)\|\leq 2\max_{z\in W(A)}|p(z)|,\qquad W(A)=\{\langle Ax,x\rangle:x\in\mathbb{C}^d,\ \|x\|_2=1\}.$$
- $$p(z)=z^k\quad\Longrightarrow\quad \|A^k\|_2\leq 2\,w(A)^k,\qquad w(A)=\max_{\|x\|_2=1}|x^*Ax|.$$
- $$h_A(\theta)=\lambda_{\max}\!\left(\frac{e^{-i\theta}A+e^{i\theta}A^*}{2}\right),\qquad \widehat w(A)=\max_{\theta\in\Theta}h_A(\theta).$$
- $$\mathcal{L}_{\mathrm{NR}}(A)=\lambda\,[\widehat w(A)-\rho]_+^2+\mu\max_{1\leq k\leq K}[2\widehat w(A)^k-\tau_k]_+^2.$$

## Implementation notes

Integrate this into the transition-matrix path of a linear state-space model or an Elman RNN. Let A in C^(d x d) be the learned recurrent matrix; for a real model, use the same formulas with conjugation removed, or represent complex arithmetic by a real 2d x 2d block matrix. The hidden-state update remains h_(t+1)=Ah_t+Bu_t. At each training step: (1) choose M=8 or 16 angles theta_j=2 pi j/M; (2) form H_j=(exp(-i theta_j)A+exp(i theta_j)A*)/2; (3) estimate lambda_max(H_j) using 3 to 5 power iterations or a differentiable Lanczos routine; (4) set w_hat=max_j lambda_max(H_j); (5) add lambda [w_hat-rho]_+^2 to the minibatch loss, with rho around 0.95 for normalized discrete-time dynamics; and (6) optionally add the multi-horizon penalty for k in {1,2,4,8}. The mathematical quantity supplied by the paper is the numerical-range polynomial bound. The angle discretization, eigenvalue approximation, rho, tau_k, and penalty weights must be selected empirically. Separately measure actual transient response by applying A^k to random probe vectors and recording max_(k<=K)||A^k v||/||v||. First test a 64-dimensional linear SSM on copy-memory or sequential-MNIST tasks against no regularizer, spectral-radius regularization, and spectral normalization, keeping parameter count and optimizer settings fixed. Success means lower worst-case hidden-state amplification, fewer exploding trajectories, and equal or better validation loss on long sequences; a useful target is a 2x reduction in measured transient amplification without slower early loss descent.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.