# Invariance-Proximity Latent Dynamics

- ID: 525
- Canonical URL: https://synthcore.org/idea/525/invariance-proximity-latent-dynamics
- API JSON: https://synthcore.org/api/idea/525.json
- API Markdown: https://synthcore.org/api/idea/525.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.01819](https://arxiv.org/abs/2607.01819)
- Category: dynamics
- Solves: stability, sample-efficiency, accuracy
- ML areas: world-model, ssm, rnn, training
- Math tags: dynamical-systems, linear-algebra, optimization, approximation-theory
- Ratings: usefulness 7/10; difficulty 5/10; novelty 6/10

## Idea description

Train an encoder and decoder whose latent observables evolve through one shared linear Koopman matrix, while directly penalizing the empirical invariance residual of the learned observable subspace. This discourages latent coordinates that fit one-step transitions but continually leave the representational subspace, improving long-horizon rollout stability.

## Mathematical statement

The paper defines Koopman invariance proximity for a subspace \(\mathbb{V}\) by \(\mathcal{I}_{\mathcal{K}}(\mathbb{V})=\sup_{g\in\mathbb{V},\|\mathcal{K}g\|\neq 0}\|\mathcal{K}g-\mathcal{K}_{\mathrm{approx}}g\|/\|\mathcal{K}g\|\). Here \(\mathcal{K}\) is the Koopman operator, with \((\mathcal{K}g)(x)=g(F(x))\) for dynamics \(x^{+}=F(x)\), \(\mathbb{V}\) is the span of the learned observables, and \(\mathcal{K}_{\mathrm{approx}}=\mathcal{P}_{\mathbb{V}}\mathcal{K}|_{\mathbb{V}}\) is the projected operator. For samples \(x^{(i)}\) and \(y^{(i)}=F(x^{(i)})\), EDMD estimates the matrix representation by \(\min_K\|\Psi_Y-K\Psi_X\|_{\mathrm{Fr}}^2\), with \(K_{\Psi}=\Psi_Y\Psi_X^{\top}(\Psi_X\Psi_X^{\top})^{\dagger}\). The columns of \(\Psi_X\) and \(\Psi_Y\) are learned observable vectors evaluated at current and next states. Use the normalized empirical residual \(\widehat{\mathcal I}=\|\Psi_Y-K\Psi_X\|_{\mathrm F}/(\|\Psi_Y\|_{\mathrm F}+\epsilon)\) as a differentiable penalty.

## Key formulas

- $$\mathcal{I}_{\mathcal{K}}(\mathbb{V}):=\sup_{g\in\mathbb{V},\|\mathcal{K}g\|\neq 0}\frac{\|\mathcal{K}g-\mathcal{K}_{\operatorname{approx}}g\|}{\|\mathcal{K}g\|}.$$
- $$\min_K\|\Psi_Y-K\Psi_X\|_{\mathrm{Fr}}^{2},\qquad K_{\Psi}=\Psi_Y\Psi_X^{\top}(\Psi_X\Psi_X^{\top})^{\dagger}.$$
- $$\widehat{\mathcal I}(\theta,K)=\frac{\|\Psi_Y(\theta)-K\Psi_X(\theta)\|_{\mathrm F}}{\|\Psi_Y(\theta)\|_{\mathrm F}+\epsilon},\qquad \mathcal L=\mathcal L_{\mathrm{recon}}+\lambda_{1}\mathcal L_{\mathrm{rollout}}+\lambda_{2}\widehat{\mathcal I}^{\,2}.$$

## Implementation notes

Integrate this into a sequential latent world model or recurrent state-space model. Let an encoder produce observables \(z_t=\psi_\theta(x_t)\in\mathbb{R}^m\), let a decoder reconstruct \(x_t\) from \(z_t\), and use a shared transition \(z_{t+1}=Kz_t\); do not use a separate unconstrained MLP transition in the first experiment. For each minibatch of adjacent pairs, form matrices \(\Psi_X=[z_0,\ldots,z_{B-1}]\) and \(\Psi_Y=[z'_1,\ldots,z'_B]\), where \(z'_t=\psi_\theta(x_{t+1})\). Compute the residual penalty exactly as \(\widehat{\mathcal I}=\|\Psi_Y-K\Psi_X\|_F/(\|\Psi_Y\|_F+10^{-6})\). The pseudocode is: encode x_t and x_{t+1}; predict zhat_next=K @ z_t; accumulate reconstruction loss, one-step loss, and lambda2 times normalized residual squared; backpropagate through encoder, decoder, and K. Initialize K every few epochs, or only at epoch zero, with the paper's EDMD estimate using ridge stabilization \(K=\Psi_Y\Psi_X^{\top}(\Psi_X\Psi_X^{\top}+\alpha I)^{-1}\); thereafter train it by gradient descent. The mathematical quantity is the residual ratio; K, the encoder parameters, and the loss weights are engineering choices. Start with a 2-layer MLP encoder, m=32, on pendulum, Lorenz, or CartPole trajectories, comparing against an equal-width nonlinear latent transition. Measure 1-, 10-, and 50-step rollout MSE, latent residual ratio, gradient explosions, and prediction error at equal parameter count. Success is lower long-horizon error and monotonic reduction of the residual ratio without sacrificing one-step accuracy.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: inconclusive
- Verdict: Built a reproducible latent-dynamics experiment with an EDMD-style normalized invariance residual and a shared linear Koopman transition, plus a nonlinear-transition baseline. The mechanism was numerically confirmed: invariant residual was 8.34e-12, the non-invariant construction gave 0.229, small injected amplitudes produced nearly linear residual scaling (R²=0.999987), and omitted-component rollout error increased from 0.04 at step 1 to 0.665 at step 20. In the learned pendulum comparison, the linear model had lower 50-step MSE (2.413 vs 2.460) but worse 1- and 10-step MSE, so there is only a mixed long-horizon signal rather than a clear overall win.

### Mechanism check

- Verdict: Built a reproducible latent-dynamics experiment with an EDMD-style normalized invariance residual and a shared linear Koopman transition, plus a nonlinear-transition baseline. The mechanism was numerically confirmed: invariant residual was 8.34e-12, the non-invariant construction gave 0.229, small injected amplitudes produced nearly linear residual scaling (R²=0.999987), and omitted-component rollout error increased from 0.04 at step 1 to 0.665 at step 20. In the learned pendulum comparison, the linear model had lower 50-step MSE (2.413 vs 2.460) but worse 1- and 10-step MSE, so there is only a mixed long-horizon signal rather than a clear overall win.
- Confidence: 8/10
- Limitations: Only one fixed random seed and one small synthetic pendulum setup were tested. The nonlinear baseline had more parameters than the linear model, no multi-seed statistical comparison was performed, and EDMD ridge initialization and broader datasets were not evaluated.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.1674
- Paired wins: 3/8
- Benchmark verdict: no significant win

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/246/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/246/bench_report.json)
- [experiment.py](https://synthcore.org/code/246/experiment.py)
- [report.md](https://synthcore.org/code/246/report.md)
- [report_bench_2026-08-30T213801.md](https://synthcore.org/code/246/report_bench_2026-08-30T213801.md)
- [rerun_output.json](https://synthcore.org/code/246/rerun_output.json)
- [results.json](https://synthcore.org/code/246/results.json)
- [Download all files as ZIP](https://synthcore.org/download/246)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.