Numerical-range regularization for nonnormal state dynamics
Implementation & benchmark of arXiv:2608.03841 — A solution to Crouzeix's conjecture
Source paper: A solution to Crouzeix's conjecture arXiv:2608.03841 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Constrain the numerical range of a learned recurrent or state-space transition matrix instead of constraining only its eigenvalues or singular norm. The resulting Crouzeix certificate controls every polynomial time filter, including multi-step powers and residual propagation, and is designed to suppress transient amplification caused by nonnormality.
Formulas
Mathematical statement
The paper's central result states that for any linear operator A on a finite-dimensional complex Hilbert space and any complex polynomial p, ||p(A)|| is at most 2 times the maximum of |p(z)| over the numerical range. Here ||.|| is the Euclidean operator norm and W(A)={x*Ax: x is a unit vector} is the numerical range. For p(z)=z^k, the result gives ||A^k||_2 <= 2 w(A)^k, where k is the rollout length and w(A)=max_{||x||_2=1}|x*Ax| is the numerical radius. The numerical-range support function is h_A(theta)=lambda_max(Re(exp(-i theta)A)), where Re(B)=(B+B*)/2 and theta is an angle. Sampling angles therefore gives an estimate of the numerical radius. The proposed neural-network surrogate penalizes [w_hat(A)-rho]_+^2, or directly penalizes predicted multi-step bounds. The theorem supplies the factor-2 bound for every polynomial; the angle grid, eigenvalue approximation, radius target rho, and rollout horizons are engineering choices.
Implementation notes
Integrate this into the transition-matrix path of a linear state-space model or an Elman RNN. Let A in C^(d x d) be the learned recurrent matrix; for a real model, use the same formulas with conjugation removed, or represent complex arithmetic by a real 2d x 2d block matrix. The hidden-state update remains h_(t+1)=Ah_t+Bu_t. At each training step: (1) choose M=8 or 16 angles theta_j=2 pi j/M; (2) form H_j=(exp(-i theta_j)A+exp(i theta_j)A*)/2; (3) estimate lambda_max(H_j) using 3 to 5 power iterations or a differentiable Lanczos routine; (4) set w_hat=max_j lambda_max(H_j); (5) add lambda [w_hat-rho]_+^2 to the minibatch loss, with rho around 0.95 for normalized discrete-time dynamics; and (6) optionally add the multi-horizon penalty for k in {1,2,4,8}. The mathematical quantity supplied by the paper is the numerical-range polynomial bound. The angle discretization, eigenvalue approximation, rho, tau_k, and penalty weights must be selected empirically. Separately measure actual transient response by applying A^k to random probe vectors and recording max_(k<=K)||A^k v||/||v||. First test a 64-dimensional linear SSM on copy-memory or sequential-MNIST tasks against no regularizer, spectral-radius regularization, and spectral normalization, keeping parameter count and optimizer settings fixed. Success means lower worst-case hidden-state amplification, fewer exploding trajectories, and equal or better validation loss on long sequences; a useful target is a 2x reduction in measured transient amplification without slower early loss descent.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.