Hyperbolic State Transition Regularization

Integrate the method into the recurrent transition matrix A in h_{t+1}=phi(Ah_t+Bx_t), or into the learned state matrix of an SSM. Start with N=64 or 128 so that a full eigendecomposition is inexpensive. At each training step: (1) compute the complex eigenvalues lambda_i of A; (2) calculate r_i=abs(lambda_i) and the gap penalty L_gap=(1/N)*sum_i*max(0,m-abs(r_i-1))^2; (3) backpropagate through the eigendecomposition when supported; and (4) optimize L_task+alpha*L_gap. Use m=0.05 and alpha values in {1e-4,1e-3,1e-2}. For larger matrices, estimate only eigenvalues nearest the unit circle with Arnoldi iteration, or parameterize A in real 2-by-2 blocks with directly controlled radii. A simpler non-differentiable alternative is a post-update projection: for every eigenvalue with radius r in [1-m,1+m], replace its radius by 1-m if r<1 and by 1+m if r>=1, preserving phase and reconstructing A. The paper supplies the spectral condition, while the margin and optimizer coefficient are empirical hyperparameters. First test a 128-unit tanh RNN against unconstrained initialization, orthogonal initialization, and spectral-radius clipping on copy-memory and permuted sequential-MNIST tasks. Inject Gaussian noise into every recurrent update and evaluate clean accuracy, noisy accuracy, hidden-state divergence over 1,000 steps, gradient norms, and seed variance. The method succeeds if it lowers long-horizon noisy loss and trajectory divergence at equal parameter count and does not substantially reduce clean-sequence accuracy.