# Nonlinearity-Subtracted Latent State-Space Model

- ID: 2238
- Canonical URL: https://synthcore.org/idea/2238/nonlinearity-subtracted-latent-state-space-model
- API JSON: https://synthcore.org/api/idea/2238.json
- API Markdown: https://synthcore.org/api/idea/2238.md
- Verification status: beats_baseline
- Source: [arXiv:2608.13373](https://arxiv.org/abs/2608.13373)
- Category: architecture
- Solves: stability, accuracy, scalability
- ML areas: ssm, rnn, world-model, training-dynamics
- Math tags: dynamical-systems, linear-algebra, control-theory, numerical-analysis
- Ratings: usefulness 8/10; difficulty 5/10; novelty 7/10

## Idea description

Build a latent continuous-time neural model with dynamics \(\dot{z}=Az+f_\phi(z)\), where \(f_\phi\) is known, separately computed, or frozen, and \(A\) is learned exclusively from the derivative residual after subtracting \(f_\phi(z)\). Parameterize \(A\) with a truncated SVD or low-rank factorization so its eigenvalues directly predict local stability and long-horizon growth.

## Mathematical statement

For latent states \(z(t_i)\in\mathbb{R}^d\), define \(Z=[z(t_1),\ldots,z(t_m)]\), \(\dot Z=[\dot z(t_1),\ldots,\dot z(t_m)]\), and \(F=[f_\phi(z(t_1)),\ldots,f_\phi(z(t_m))]\). NSDMD estimates the unknown linear operator from \(\dot z=Az+f_\phi(z)\) as \(\widehat A=(\dot Z-F)Z^\dagger\), where \(Z^\dagger\) is the Moore-Penrose pseudoinverse. For a rank-\(r\) approximation, compute \(Z=U\Sigma V^*\), retain \(U_r,\Sigma_r,V_r\), and set \(\widehat A_r=(\dot Z-F)V_r\Sigma_r^{-1}U_r^*\). The nonlinear contribution is removed before regression, preventing it from biasing the identified linearization when \(F\) is correct. The linear subsystem is locally asymptotically stable when every eigenvalue of \(\widehat A_r\) has negative real part; under a timestep \(\Delta t\), its discrete flow is stable when \(\rho(\exp(\widehat A_r\Delta t))<1\).

## Key formulas

- $$\dot z(t)=Az(t)+f_\phi(z(t))$$
- $$\widehat A=(\dot Z-F)Z^\dagger$$
- $$Z=U\Sigma V^*,\qquad \widehat A_r=(\dot Z-F)V_r\Sigma_r^{-1}U_r^*$$
- $$\rho\!\left(e^{\widehat A_r\Delta t}\right)=e^{\Delta t\max_j\operatorname{Re}\lambda_j(\widehat A_r)}<1\quad\Longleftrightarrow\quad\max_j\operatorname{Re}\lambda_j(\widehat A_r)<0$$

## Implementation notes

1. Integration point: use this transition model inside a latent neural ODE, recurrent state-space model, or world model. An encoder maps observations to latent states \(z_i\), a prescribed or pretrained module computes \(f_\phi(z_i)\), and only the linear operator \(A\) is fitted by NSDMD. Use a dense \(d\times d\) matrix for small latent dimension or a rank-constrained matrix for larger systems. 2. Pseudocode: encode trajectories into \(Z\); estimate \(\dot Z\) using automatic differentiation when observations are differentiable, otherwise use centered finite differences; compute \(F=f_\phi(Z)\); calculate the thin SVD \(Z=U\Sigma V^T\); retain rank \(r\); update \(A\leftarrow(\dot Z-F)V_r\Sigma_r^{-1}U_r^T\); integrate \(\dot z=Az+f_\phi(z)\) with an ODE solver. During joint training, alternate this closed-form update with gradient updates to the encoder and nonlinear module, using \(\|\dot Z-AZ-F\|_F^2\) as the residual loss. 3. Computed versus estimated: \(F\) is computed by the known nonlinear block; \(\dot Z\) is estimated from sampled trajectories; \(A\) is computed by pseudoinverse regression; rank is selected from the singular-value spectrum. For guaranteed linear stability, parameterize \(A=S-KK^T\), with symmetric \(S\), or reject updates with positive maximum real eigenvalue. 4. First cheap experiment: use the Lorenz system with exact nonlinear term \(f(x)=(0,-xz,xy)^T\), comparing ordinary vector-field regression, a generic neural ODE, and NSDMD. Train on one trajectory and evaluate held-out long-horizon rollouts, linear-operator error, and eigenvalues around the mean state. The falsifiable prediction is that NSDMD has lower linear-operator error as nonlinear amplitude increases, while unsplit regression becomes biased. For a stable benchmark, measured \(\rho(e^{\widehat A\Delta t})\) should be below one and within approximately 20% of the true spectral radius. Increasing rank should reduce error until an identifiable-subspace plateau appears.

## Verification

- Status: beats_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: helps
- Verdict: Built a NumPy NSDMD MVP with full and truncated-SVD estimators, stability checks, nonlinear-amplitude sweeps, rank sweeps, and a noisy baseline comparison. The mechanism manifested: correct subtraction recovered A to 1.17e-15 relative error across amplitudes, unsplit bias scaled linearly with observed slope 0.1321579163 versus predicted 0.1321579163, rank residual decreased monotonically to numerical zero at full rank, and the spectral-radius identity matched exactly. In the noisy mini-test, NSDMD improved A error from 0.2651 to 0.00825 and test MSE from 0.01041 to 9.95e-06.

### Mechanism check

- Verdict: Built a NumPy NSDMD MVP with full and truncated-SVD estimators, stability checks, nonlinear-amplitude sweeps, rank sweeps, and a noisy baseline comparison. The mechanism manifested: correct subtraction recovered A to 1.17e-15 relative error across amplitudes, unsplit bias scaled linearly with observed slope 0.1321579163 versus predicted 0.1321579163, rank residual decreased monotonically to numerical zero at full rank, and the spectral-radius identity matched exactly. In the noisy mini-test, NSDMD improved A error from 0.2651 to 0.00825 and test MSE from 0.01041 to 9.95e-06.
- Confidence: 9/10
- Limitations: Only a synthetic 5-dimensional algebraic toy system was tested, with known exact nonlinear terms and directly observed states. It did not test learned latent encoders, estimated derivatives from trajectories, adaptive rank selection, ODE integration error, model mismatch in f_phi, or a Lorenz/real-world benchmark.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 8/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/793/bench_report.json)
- [nsdmd_experiment.py](https://synthcore.org/code/793/nsdmd_experiment.py)
- [report.md](https://synthcore.org/code/793/report.md)
- [report_bench_2026-09-01T104314.md](https://synthcore.org/code/793/report_bench_2026-09-01T104314.md)
- [results.json](https://synthcore.org/code/793/results.json)
- [stage2_bench.py](https://synthcore.org/code/793/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/793)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.