Gramian-Regularized Latent State Models

Integrate this into a GRU, continuous-time neural ODE, or state-space model with latent state $z_t$. For each training window, linearize the transition and readout around the current rollout: compute $A_t=\partial f_\theta/\partial z$, $B_t=\partial f_\theta/\partial u$, and $C_t=\partial h_\theta/\partial z$. Initialize $\Phi=I$, $W_o=0$, and $W_r=0$. For each timestep, update $\Phi_{t+1}=(I+\Delta t A_t)\Phi_t$ and accumulate $W_o\mathrel{+}=\Delta t\,\Phi_t^TC_t^TC_t\Phi_t$. Use backward transition propagation $\Psi_t=\Psi_{t+1}(I+\Delta t A_t)$ to accumulate $W_r\mathrel{+}=\Delta t\,\Psi_{t+1}B_tB_t^T\Psi_{t+1}^T$. Estimate minimum eigenvalues with 5--10 Lanczos or power iterations instead of a full eigendecomposition. Add the normalized penalties to the task loss and clip gradients from the Gramian term. The paper supplies the finite-time Gramian mechanism; neural Jacobians and finite-window approximations are estimated from minibatch rollouts. First experiment: compare a 32-dimensional GRU with and without the penalties on permuted-MNIST and a synthetic delayed-copy task. Sweep $\lambda_o$ and $\lambda_r$. The predicted signature is a sharp performance degradation when normalized $\lambda_{\min}(W_o)$ falls below approximately $\epsilon=0.05$; regularization should prevent this collapse. Near the weak-observability boundary, latent reconstruction error should scale approximately as $1/\lambda_{\min}(W_o)$.