Invariant-Preserving Latent Compression

(1) Integration point: use this in a latent SSM, neural operator, or diffusion/world-model transition whose hidden physical state is compressed to rank $r$. Let the flattened state be $z_n\in\mathbb{R}^N$, let the transition produce $y_{n+1}=F_\theta(z_n,u_n,\xi_n)$, and let $B$ contain known invariant directions. For a kinetic grid, columns can be discrete mass, each momentum component, and energy weights; for a generic system, use conserved charges, total mass, probability, or prescribed linear diagnostics. (2) Pseudocode: precompute full-rank $B$ and $G=(B^TB)^{-1}$. At each step compute y = transition_theta(z, input, noise). If using a rank-$r$ state representation, maintain $U_r$, form Q = qr(concat(U_r, B)).Q, and replace the state or update subspace by Q Q.T y; when a strict target is required, set z = y + B @ G @ (c_next - B.T @ y). Recompress only after augmentation: compute an SVD or randomized SVD in the orthogonal complement of $B$, retain the top $r-m$ residual directions, then concatenate them with $B$ and orthonormalize. (3) Compute $B$ and the correction matrix analytically from the chosen statistics. Estimate only the residual basis and singular values empirically; use QR plus randomized SVD for large $N$. If invariants are unknown, first fit candidate linear statistics on simulated trajectories and retain only those with low empirical variance. (4) First experiment: train a rank-16 latent recurrent neural operator on a 2-D advection or particle-grid dataset with a known conserved total mass and momentum, comparing ordinary rank-16 SVD compression against augmented rank-16 compression with $m=3$ invariant columns. Measure long-horizon rollout error, invariant drift, memory, and wall-clock cost. Success means invariant drift is reduced by at least 10x and long-horizon error improves at equal rank, with less than 15% additional transition cost; a stronger result is matching the unconstrained model's short-horizon error while using a smaller rank.