Continuous Ergodic Projection for Recurrent States

Use an RNN, state-space model, or learned world model with latent update $z_{t+1}=T_\theta(z_t,u_t)$. Draw two nearby initial latent states $z_0,z'_0$, roll each for $N$ steps, and construct an empirical subprobability measure $\widehat E(z_0)=\widehat m(z_0)\sum_{t=0}^{N-1}w_t\delta_{z_t}$, with $w_t\ge0$ and $\sum_t w_t=1$. A bounded rollout should have mass near one; estimate an escape score $R(z_0)=N^{-1}\sum_t\|z_t\|^2$ or terminal growth rate, and train a mass head toward $\widehat m\approx\exp(-\alpha R)$ after latent-scale normalization. Compute projected features $P_\theta f(z_0)=\widehat m\sum_tw_tf_\phi(z_t)$ and add task loss, $\lambda_c\mathcal L_{\mathrm{cont}}$, and $\lambda_e\mathcal L_{\mathrm{escape}}$. Pseudocode: roll out both states; calculate weights, mass, and projected features; evaluate task loss; calculate a differentiable kernel approximation to $W_1$; backpropagate the total loss. The paper supplies the invariant-projection and continuity mechanism; finite-horizon weights, escape scores, and mass targets are empirical approximations. First test a GRU on Lorenz-63 or a damped oscillator, comparing ordinary teacher forcing with this regularizer. Measure projection residual $\|A_Nf-P_\theta f\|$ and the continuity ratio $W_1(\widehat E(z),\widehat E(z'))/(\|z-z'\|+\varepsilon)$. The prediction is that bounded trajectories have decreasing residual, approximately $O(N^{-1/2})$ under mixing, while escaping trajectories have decreasing mass. On a known uniquely ergodic rotation, the learned projection should match the analytic invariant mean within 10-20%, and the averaging residual should decrease monotonically with horizon.