✓✓ Beats tuned baseline 2026

Nonlinearity-Subtracted Latent State-Space Model

Implementation & benchmark of arXiv:2608.13373 — Data-driven linear analysis of dynamical systems via nonlinearity-subtracted dynamic mode decomposition

Usefulness8/10

Difficulty5/10

Novelty7/10

Architecture Stability Accuracy Scalability Ssm Rnn World model Training dynamics Dynamical systems Linear algebra Control theory Numerical analysis

Source paper: Data-driven linear analysis of dynamical systems via nonlinearity-subtracted dynamic mode decomposition arXiv:2608.13373 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Build a latent continuous-time neural model with dynamics $\dot{z}=Az+f_\phi(z)$, where $f_\phi$ is known, separately computed, or frozen, and $A$ is learned exclusively from the derivative residual after subtracting $f_\phi(z)$. Parameterize $A$ with a truncated SVD or low-rank factorization so its eigenvalues directly predict local stability and long-horizon growth.

Formulas

$$\dot z(t)=Az(t)+f_\phi(z(t))$$

$$\widehat A=(\dot Z-F)Z^\dagger$$

$$Z=U\Sigma V^*,\qquad \widehat A_r=(\dot Z-F)V_r\Sigma_r^{-1}U_r^*$$

$$\rho\!\left(e^{\widehat A_r\Delta t}\right)=e^{\Delta t\max_j\operatorname{Re}\lambda_j(\widehat A_r)}<1\quad\Longleftrightarrow\quad\max_j\operatorname{Re}\lambda_j(\widehat A_r)<0$$

Mathematical statement

For latent states $z(t_i)\in\mathbb{R}^d$, define $Z=[z(t_1),\ldots,z(t_m)]$, $\dot Z=[\dot z(t_1),\ldots,\dot z(t_m)]$, and $F=[f_\phi(z(t_1)),\ldots,f_\phi(z(t_m))]$. NSDMD estimates the unknown linear operator from $\dot z=Az+f_\phi(z)$ as $\widehat A=(\dot Z-F)Z^\dagger$, where $Z^\dagger$ is the Moore-Penrose pseudoinverse. For a rank-$r$ approximation, compute $Z=U\Sigma V^*$, retain $U_r,\Sigma_r,V_r$, and set $\widehat A_r=(\dot Z-F)V_r\Sigma_r^{-1}U_r^*$. The nonlinear contribution is removed before regression, preventing it from biasing the identified linearization when $F$ is correct. The linear subsystem is locally asymptotically stable when every eigenvalue of $\widehat A_r$ has negative real part; under a timestep $\Delta t$, its discrete flow is stable when $\rho(\exp(\widehat A_r\Delta t))<1$.

Implementation notes

1. Integration point: use this transition model inside a latent neural ODE, recurrent state-space model, or world model. An encoder maps observations to latent states $z_i$, a prescribed or pretrained module computes $f_\phi(z_i)$, and only the linear operator $A$ is fitted by NSDMD. Use a dense $d\times d$ matrix for small latent dimension or a rank-constrained matrix for larger systems. 2. Pseudocode: encode trajectories into $Z$; estimate $\dot Z$ using automatic differentiation when observations are differentiable, otherwise use centered finite differences; compute $F=f_\phi(Z)$; calculate the thin SVD $Z=U\Sigma V^T$; retain rank $r$; update $A\leftarrow(\dot Z-F)V_r\Sigma_r^{-1}U_r^T$; integrate $\dot z=Az+f_\phi(z)$ with an ODE solver. During joint training, alternate this closed-form update with gradient updates to the encoder and nonlinear module, using $\|\dot Z-AZ-F\|_F^2$ as the residual loss. 3. Computed versus estimated: $F$ is computed by the known nonlinear block; $\dot Z$ is estimated from sampled trajectories; $A$ is computed by pseudoinverse regression; rank is selected from the singular-value spectrum. For guaranteed linear stability, parameterize $A=S-KK^T$, with symmetric $S$, or reject updates with positive maximum real eigenvalue. 4. First cheap experiment: use the Lorenz system with exact nonlinear term $f(x)=(0,-xz,xy)^T$, comparing ordinary vector-field regression, a generic neural ODE, and NSDMD. Train on one trajectory and evaluate held-out long-horizon rollouts, linear-operator error, and eigenvalues around the mean state. The falsifiable prediction is that NSDMD has lower linear-operator error as nonlinear amplitude increases, while unsplit regression becomes biased. For a stable benchmark, measured $\rho(e^{\widehat A\Delta t})$ should be below one and within approximately 20% of the true spectral radius. Increasing rank should reduce error until an identifiable-subspace plateau appears.

Verification

Beats tuned baseline

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significant win ✓

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a NumPy NSDMD MVP with full and truncated-SVD estimators, stability checks, nonlinear-amplitude sweeps, rank sweeps, and a noisy baseline comparison. The mechanism manifested: correct subtraction recovered A to 1.17e-15 relative error across amplitudes, unsplit bias scaled linearly with observed slope 0.1321579163 versus predicted 0.1321579163, rank residual decreased monotonically to numerical zero at full rank, and the spectral-radius identity matched exactly. In the noisy mini-test, NSDMD improved A error from 0.2651 to 0.00825 and test MSE from 0.01041 to 9.95e-06.

Agent confidence: 9/10

Baseline: Noisy unsplit regression: relative A error 0.2650713; held-out vector-field MSE 0.0104145
Idea: Noisy nonlinearity-subtracted regression: relative A error 0.00825294; held-out vector-field MSE 0.00000995110; noiseless max A error across nonlinear amplitudes 1.17e-15; amplitude-bias slope observed/predicted 0.1321579163/0.1321579163; stability-radius max formula error 0.0

Limitations: Only a synthetic 5-dimensional algebraic toy system was tested, with known exact nonlinear terms and directly observed states. It did not test learned latent encoders, estimated derivatives from trajectories, adaptive rank selection, ODE integration error, model mismatch in f_phi, or a Lorenz/real-world benchmark.

How to run: python3 nsdmd_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.13373) Sep 2, 2026

Beats the tuned baseline by 74.2% (p=0.0081, wins 8 / 8 paired seeds; metric: lower is better).

Benchmark: Dynamics forecasting
Model: residual_euler_shared
Paired seeds: 8
Baseline mean: 0.001
Idea mean: 0.0003
Effect (Δ): -0.0008 (−74.2%; negative = idea better)
Wins: 8 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±44.3%
Mechanism: Confirmed ✓
Practical effect: Helps
Baseline tuning: swept over 3 configs

Limitations:

Tested only the built-in actuated-pendulum dynamics track and a small residual-Euler model, not learned latent encoders, continuous-time ODE integration, adaptive rank selection, Lorenz trajectories, model mismatch in the prescribed nonlinear term, or larger latent dimensions. The mechanism signature was measured on one trained idea model seed rather than independently aggregated over all eight seeds.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Worked ✓

Agent confidence: 9/10

Implemented the Nonlinearity-Subtracted Latent State-Space intervention on the structurally matched dynamics track using an identical residual-Euler architecture and equal training budgets. Across 8 paired seeds, the idea reduced test MSE from 0.00101209 to 0.000260839, with delta_mean=-0.000751255 and permutation p=0.0081; therefore the benchmark verdict is idea better (significant). A trained-model behavior probe found subtraction reduced relative linear-operator error from 0.57682 to 0.42079 (1.37x improvement), confirming the proposed mechanism on the neural benchmark.

Baseline: Dynamics residual-Euler shared model, learned nonlinear block, lr=0.01: test MSE mean 0.0010120938641193788, std 0.000439246073070232, per-seed [0.000399811688112095, 0.0012618446489796042, 0.0005643676850013435, 0.001626465586014092, 0.0009226426482200623, 0.0016466405941173434, 0.0006691205780953169, 0.0010058574844151735]
Idea: Dynamics residual-Euler shared model, prescribed frozen pendulum nonlinearity, lr=0.01: test MSE mean 0.000260838560279808, std 0.00022691108449835587, per-seed [0.0002579344727564603, 0.0001047899349941872, 0.00020367394608911127, 0.00016455352306365967, 0.0008300704648718238, 0.00030808275914750993, 7.328044011956081e-05, 0.00014432294119615108]
Mechanism signature: nonlinear energy ratio 0.2191952; unsplit relative A error 0.5768200; subtracted relative A error 0.4207863; improvement factor 1.3708146; confirmed=true
Paired delta: -0.0007512553038395708; idea wins 8/8; permutation p=0.0081

Benchmark result