P-Metric Contractive State Transition

Integrate this at the recurrent or state-space transition, not at the readout: for an RNN use h_{t+1}=f_theta(h_t,u_t), and for a latent world model use z_{t+1}=f_theta(z_t,a_t). Start with P=I for a cheap baseline, then optionally parameterize P=LL^T+epsilon I and normalize trace(P)=d to prevent the penalty from being defeated by metric scaling. For each minibatch, collect hidden states x from teacher-forced or replay trajectories. Estimate the largest singular value of the transformed Jacobian T=P^(1/2)J_theta(x)P^(-1/2) by 2-5 power iterations using Jacobian-vector products and vector-Jacobian products; do not materialize J. Use this update: inputs are x, target eta, metric P, and coefficient lambda; initialize a random unit vector v; repeatedly compute w=T v, set v=w/(||w||_2+epsilon); after the iterations set s=||T v||_2; accumulate hinge=(max(0,s^2-eta^2))^2; optimize task_loss+lambda*mean(hinge). Detach the power-iteration vector between iterations initially for stability, then test full differentiation. The paper supplies the contraction inequality and target factor; the Jacobian norm, sampled states, and any P update are estimated numerically. Begin with a 1-2 layer GRU on sequential MNIST or a small linear-control-inspired latent prediction task, comparing an ordinary GRU, spectral-normalized GRU, and this penalty at equal parameter count and FLOPs. Measure validation accuracy, gradient norms, multi-step rollout error, sensitivity to perturbed initial states, and the fraction of sampled states violating the bound. Success means fewer exploding or unstable rollouts and lower long-horizon prediction error at comparable one-step loss, without materially reducing short-horizon accuracy.