△ Mechanism confirmed, baseline not beaten 2026

Invariance-Proximity Latent Dynamics

Implementation & benchmark of arXiv:2607.01819 — Koopman operator theory: fundamentals, control, and applications

Usefulness7/10

Difficulty5/10

Novelty6/10

Dynamics Stability Sample efficiency Accuracy World model Ssm Rnn Training Dynamical systems Linear algebra Optimization Approximation theory

Source paper: Koopman operator theory: fundamentals, control, and applications arXiv:2607.01819 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Train an encoder and decoder whose latent observables evolve through one shared linear Koopman matrix, while directly penalizing the empirical invariance residual of the learned observable subspace. This discourages latent coordinates that fit one-step transitions but continually leave the representational subspace, improving long-horizon rollout stability.

Formulas

$$\mathcal{I}_{\mathcal{K}}(\mathbb{V}):=\sup_{g\in\mathbb{V},\|\mathcal{K}g\|\neq 0}\frac{\|\mathcal{K}g-\mathcal{K}_{\operatorname{approx}}g\|}{\|\mathcal{K}g\|}.$$

$$\min_K\|\Psi_Y-K\Psi_X\|_{\mathrm{Fr}}^{2},\qquad K_{\Psi}=\Psi_Y\Psi_X^{\top}(\Psi_X\Psi_X^{\top})^{\dagger}.$$

$$\widehat{\mathcal I}(\theta,K)=\frac{\|\Psi_Y(\theta)-K\Psi_X(\theta)\|_{\mathrm F}}{\|\Psi_Y(\theta)\|_{\mathrm F}+\epsilon},\qquad \mathcal L=\mathcal L_{\mathrm{recon}}+\lambda_{1}\mathcal L_{\mathrm{rollout}}+\lambda_{2}\widehat{\mathcal I}^{\,2}.$$

Mathematical statement

The paper defines Koopman invariance proximity for a subspace $\mathbb{V}$ by $\mathcal{I}_{\mathcal{K}}(\mathbb{V})=\sup_{g\in\mathbb{V},\|\mathcal{K}g\|\neq 0}\|\mathcal{K}g-\mathcal{K}_{\mathrm{approx}}g\|/\|\mathcal{K}g\|$. Here $\mathcal{K}$ is the Koopman operator, with $(\mathcal{K}g)(x)=g(F(x))$ for dynamics $x^{+}=F(x)$, $\mathbb{V}$ is the span of the learned observables, and $\mathcal{K}_{\mathrm{approx}}=\mathcal{P}_{\mathbb{V}}\mathcal{K}|_{\mathbb{V}}$ is the projected operator. For samples $x^{(i)}$ and $y^{(i)}=F(x^{(i)})$, EDMD estimates the matrix representation by $\min_K\|\Psi_Y-K\Psi_X\|_{\mathrm{Fr}}^2$, with $K_{\Psi}=\Psi_Y\Psi_X^{\top}(\Psi_X\Psi_X^{\top})^{\dagger}$. The columns of $\Psi_X$ and $\Psi_Y$ are learned observable vectors evaluated at current and next states. Use the normalized empirical residual $\widehat{\mathcal I}=\|\Psi_Y-K\Psi_X\|_{\mathrm F}/(\|\Psi_Y\|_{\mathrm F}+\epsilon)$ as a differentiable penalty.

Implementation notes

Integrate this into a sequential latent world model or recurrent state-space model. Let an encoder produce observables $z_t=\psi_\theta(x_t)\in\mathbb{R}^m$, let a decoder reconstruct $x_t$ from $z_t$, and use a shared transition $z_{t+1}=Kz_t$; do not use a separate unconstrained MLP transition in the first experiment. For each minibatch of adjacent pairs, form matrices $\Psi_X=[z_0,\ldots,z_{B-1}]$ and $\Psi_Y=[z'_1,\ldots,z'_B]$, where $z'_t=\psi_\theta(x_{t+1})$. Compute the residual penalty exactly as $\widehat{\mathcal I}=\|\Psi_Y-K\Psi_X\|_F/(\|\Psi_Y\|_F+10^{-6})$. The pseudocode is: encode x_t and x_{t+1}; predict zhat_next=K @ z_t; accumulate reconstruction loss, one-step loss, and lambda2 times normalized residual squared; backpropagate through encoder, decoder, and K. Initialize K every few epochs, or only at epoch zero, with the paper's EDMD estimate using ridge stabilization $K=\Psi_Y\Psi_X^{\top}(\Psi_X\Psi_X^{\top}+\alpha I)^{-1}$; thereafter train it by gradient descent. The mathematical quantity is the residual ratio; K, the encoder parameters, and the loss weights are engineering choices. Start with a 2-layer MLP encoder, m=32, on pendulum, Lorenz, or CartPole trajectories, comparing against an equal-width nonlinear latent transition. Measure 1-, 10-, and 50-step rollout MSE, latent residual ratio, gradient explosions, and prediction error at equal parameter count. Success is lower long-horizon error and monotonic reduction of the residual ratio without sacrificing one-step accuracy.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a reproducible latent-dynamics experiment with an EDMD-style normalized invariance residual and a shared linear Koopman transition, plus a nonlinear-transition baseline. The mechanism was numerically confirmed: invariant residual was 8.34e-12, the non-invariant construction gave 0.229, small injected amplitudes produced nearly linear residual scaling (R²=0.999987), and omitted-component rollout error increased from 0.04 at step 1 to 0.665 at step 20. In the learned pendulum comparison, the linear model had lower 50-step MSE (2.413 vs 2.460) but worse 1- and 10-step MSE, so there is only a mixed long-horizon signal rather than a clear overall win.

Agent confidence: 8/10

Baseline: Nonlinear latent transition: parameters=1266, one-step MSE=0.03747, rollout MSE steps 1/10/50=0.06062/0.33295/2.46007, residual=0.26542
Idea: Linear shared transition with normalized invariance penalty: parameters=778, one-step MSE=0.04234, rollout MSE steps 1/10/50=0.17125/0.77683/2.41329, residual=0.30296; toy residual scaling R²=0.999987

Limitations: Only one fixed random seed and one small synthetic pendulum setup were tested. The nonlinear baseline had more parameters than the linear model, no multi-seed statistical comparison was performed, and EDMD ridge initialization and broader datasets were not evaluated.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.01819) Sep 2, 2026

No significant difference vs. the tuned baseline (+7.3%, p=0.1674, wins 3 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0013
Idea mean: 0.0014
Effect (Δ): 0.0001 (+7.3%; negative = idea better)
Wins: 3 / 8 paired seeds
p-value: 0.1674 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±10.9%
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 6 configs

Limitations:

The full valid protocol ran on CPU because the initial CUDA attempt exited with code -4. It used 400 training samples, 200 test samples, 15 epochs, six baseline configurations and three idea configurations. No larger datasets, longer training, alternate dynamics tracks, or rollout-specific primary metric were tested; the mechanism signature was measured on seed 0 while task metrics used all eight paired seeds.

How to run: CUDA_VISIBLE_DEVICES='' /home/maxwelhelp/main/bin/python3 bench_experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 30, 2026 latest Failed ✗

Agent confidence: 10/10

The Invariance-Proximity Latent Dynamics intervention was evaluated on the registered built-in dynamics track with identical rnn_small GRU systems, differing only by the added normalized latent invariance residual loss. The idea's test MSE was worse than the tuned baseline (0.0014470 vs 0.0013484), with paired delta +0.00009856 and permutation p=0.1674, so it is not a significant win. The trained-model mechanism signature was confirmed: shared-K latent residual fell from 0.1881 to 0.0579 (69.2% reduction), but this did not transfer to better task performance.

Baseline: mean test MSE 0.0013484373, std 0.0002926706, best_cfg lr=0.003 weight_decay=0.0001
Idea: mean test MSE 0.0014469989, std 0.0002735543, selected cfg lr=0.003 weight_decay=0.0 lambda=0.6
Paired delta: +0.0000985617; idea wins 3/8; permutation p=0.1674

Benchmark result