Mean-Square Stable Neural Recurrence

Integrate this into a small GRU-free linear RNN or diagonal state-space layer first, then extend it to a nonlinear RNN using the Jacobian with respect to the hidden state. Represent stochasticity as M_t=M_0+sum_i xi_{i,t}M_i. For dropout-like channel noise, M_i is the contribution of one perturbation channel; for multiplicative parameter noise, obtain M_i by differentiating the recurrent matrix with respect to selected parameter groups. At each training interval, sample a minibatch of hidden states, compute M_0 and M_i with automatic differentiation, form K implicitly, and estimate its largest eigenvalue using 10-30 power iterations. Add lambda max(0,rho_hat(K)-(1-epsilon))^2 to the task loss. Initially stop gradients through the power-iteration vector; later use a differentiable eigensolver for small hidden dimensions. The covariance recursion and spectral condition come from the paper; local Jacobians, noise covariance, and rho are estimated empirically. Avoid explicit d^2-by-d^2 matrices using randomized Kronecker-vector products. First experiment: train a 32-unit linear RNN and tanh RNN on adding-two-numbers or sequential MNIST with Gaussian multiplicative recurrent-weight noise. Sweep noise variance and record hidden covariance for 1,000 inference steps. The predicted signature is a divergence boundary when rho_hat(K) crosses 1: below it covariance approaches a finite plateau with geometric decay rate approximately rho_hat(K), while above it trace covariance grows exponentially. The measured transition should agree within 20%.