Data-driven invariant hidden-state ellipsoid

1. Integration point: use a GRU-free residual RNN or linear state-space block, for example $h_{t+1}=\phi(W_hh_t+W_zz_t+b)$, and begin with the linearized or residual version $h_{t+1}=Ah_t+Bz_t+e_t$. Collect hidden trajectories from a normally trained baseline over calibration sequences. Fit $A,B$ by ridge regression and compute residuals $e_t=h_{t+1}-Ah_t-Bz_t$. Estimate $Q_e$ as an inflated covariance ellipsoid or a diagonal ellipsoid containing 99 percent of residuals; estimate $Q_\xi$ from the feature distribution. 2. Pseudocode: fit $A,B$; initialize $P\succ0$, $K$, and positive multipliers $\alpha_i$; solve an SDP or alternating convex program minimizing $\log\det P$ subject to $FPF^T/\alpha_0+Q_e/\alpha_1+BQ_\xi B^T/\alpha_2\preceq P$, with $F=A+BK$ and a norm bound on $K$; freeze the resulting $P,K$ for the first experiment. During training add $\lambda\max(0,h_t^TP^{-1}h_t-1)^2$ to the task loss. At inference optionally project $h_t$ to $\mathcal{E}(P)$ using $h_t\leftarrow h_t/\max(1,\sqrt{h_t^TP^{-1}h_t})$. 3. Computed from the paper's mechanism: the invariant-set inclusion and SDP certificate. Estimated empirically: $A,B$, residual ellipsoid $Q_e$, feature bound $Q_\xi$, and nonlinear approximation error. For nonlinear activations, enlarge $Q_e$ using held-out Jacobian and residual errors; the guarantee is empirical unless a global Lipschitz bound is supplied. 4. First cheap experiment: train a small tanh RNN on sequential MNIST, permuted MNIST, or a synthetic long-horizon copy task, comparing the baseline, ellipsoid regularization, and projection. Sweep input-noise scale and recurrent spectral scale. Measure $q$, ellipsoid violation rate, hidden norm, gradient norm, and loss after 1,000 to 10,000 recurrent steps. The predicted signature is a sharp degradation when $q$ crosses one: for $q<1$, violations and hidden energy remain bounded and the disturbance-induced radius follows approximately $1/(1-q)$; for $q>1$, hidden norms grow exponentially at a rate close to $\log q$. The estimated transition should agree with the observed divergence boundary within approximately 20 percent.