# Spectral-submanifold latent dynamics

- ID: 698
- Canonical URL: https://synthcore.org/idea/698/spectral-submanifold-latent-dynamics
- API JSON: https://synthcore.org/api/idea/698.json
- API Markdown: https://synthcore.org/api/idea/698.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.10675](https://arxiv.org/abs/2607.10675)
- Category: dynamics
- Solves: stability, accuracy, memory
- ML areas: rnn, ssm, world-model, training-dynamics
- Math tags: dynamical-systems, spectral-theory, pde, bifurcations, control-theory
- Ratings: usefulness 8/10; difficulty 6/10; novelty 7/10

## Idea description

Replace an unconstrained high-dimensional recurrent hidden state with a low-dimensional nonlinear invariant manifold attached to a selected spectral subspace of the hidden-state linearization. Learn both the manifold graph and its reduced nonlinear dynamics, then roll out the reduced coordinates for long horizons while reconstructing the full hidden state only when needed.

## Mathematical statement

The source studies the forced-damped second-order evolution equation $\ddot u+B\dot u+Au=f(u,\dot u)+\varepsilon h(t)$, where $u$ is the PDE state, $A$ is the elastic operator, $B$ is the damping operator, $f$ is a nonlinear term, $h$ is external forcing, and $\varepsilon$ is forcing amplitude. Convert a neural hidden-state system to first order, $\dot x=F(x,t)$, and let $E$ be the span of selected eigenvectors of the Jacobian $J=D_xF(0)$ with eigenvalues $\lambda_1,\ldots,\lambda_d$. An SSM is represented by a nonlinear graph $x=W(z)=Vz+H(z)$, where $z\in\mathbb R^d$, $V$ contains the selected eigenvectors, and $H(z)=O(\|z\|^2)$ lies transverse to $E$. Its defining property is the invariance equation $D_zW(z)R(z)=F(W(z),t)$, where $R$ is the reduced latent vector field. Enforce this residual during training. For a two-dimensional oscillatory mode with complex coordinate $z$, use the normal form $\dot z=(\lambda+i\omega)z+c z|z|^2+\varepsilon q(t)$, with damping rate $\lambda<0$, linear frequency $\omega$, cubic coefficient $c=c_r+ic_i$, and forcing projection $q$. The predicted amplitude decay and backbone frequency are $\dot r=\lambda r+c_r r^3$ and $\Omega(r)=\omega+c_i r^2$, respectively, for amplitude $r=|z|$.

## Key formulas

- $$\ddot u+B\dot u+Au=f(u,\dot u)+\varepsilon h(t)$$
- $$x=W(z)=Vz+H(z),\qquad H(z)=O(\|z\|^2),\qquad D_zW(z)R(z)=F(W(z),t)$$
- $$\dot z=(\lambda+i\omega)z+c z|z|^2+\varepsilon q(t),\qquad \dot r=\lambda r+c_r r^3,\qquad \Omega(r)=\omega+c_i r^2$$

## Implementation notes

1. Integration point: use a GRU, continuous-time RNN, or latent neural ODE with hidden state $x\in\mathbb R^n$. Add an encoder into $d=2$ or $4$ modal coordinates, a decoder $W_\theta(z)=V_\theta z+H_\theta(z)$, and a reduced vector field $R_\phi(z)$. Use $z$, rather than $x$, for long-horizon rollout. Estimate the hidden Jacobian $J$ at a reference state with automatic differentiation and initialize $V_\theta$ from eigenvectors associated with slow stable complex-conjugate eigenvalues. 2. Pseudocode: encode $z=E_\theta(x)$; reconstruct $\hat x=W_\theta(z)$; evaluate $\hat F=F_\theta(\hat x,t)$ and $\hat R=R_\phi(z,t)$; calculate $e_{\mathrm{inv}}=D_zW_\theta(z)\hat R-\hat F$; optimize $L=L_{\mathrm{prediction}}+\alpha\|e_{\mathrm{inv}}\|^2+\beta\|E_\theta(W_\theta(z))-z\|^2$. At inference, integrate $\dot z=R_\phi(z,t)$ and decode at output times. 3. Computed from the paper's mechanism: the graph invariance equation, spectral-subspace initialization, and reduced amplitude/frequency structure. Estimated empirically: $J$, $\lambda$, $\omega$, $c_r$, and $c_i$; fit these coefficients by least squares from latent trajectories after training. 4. First cheap experiment: train on a damped Duffing oscillator or a two-mode nonlinear beam surrogate, comparing a linear projection, a standard autoencoder latent ODE, and the SSM model at equal latent dimension. Sweep initial amplitude and measure frequency from zero crossings, amplitude decay, invariance residual, and 50- to 200-step rollout error. The quantitative prediction is that $\Omega(r)$ is approximately affine in $r^2$ with slope $c_i$, while amplitude follows the cubic-corrected decay equation. Rollout error should rise sharply as an omitted eigenvalue approaches the selected spectral band, when the spectral-gap ratio $\min_{j\notin E}|\operatorname{Re}\lambda_j|/\max_{i\in E}|\operatorname{Re}\lambda_i|$ approaches one.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a reproducible nonlinear SSM toy with a quadratic graph, cubic reduced dynamics, coefficient fitting, invariance checking, spectral-gap sweep, and rollout comparison. The backbone prediction Ω(r)=ω+c_i r² was observed as intercept 2.000000 and slope 0.350000 versus predicted 2.0 and 0.35; the radial law recovered λ=-0.080000 and c_r=-0.120000 versus the exact values. Transverse decay matched every swept γ from 0.04 to 0.64 within about 1e-9, and cubic SSM rollout RMSE was lower than the linear baseline at horizons 5/10/18 (0.0699/0.0890/0.0966 vs 0.3397/0.3833/0.3652).

### Mechanism check

- Verdict: Built a reproducible nonlinear SSM toy with a quadratic graph, cubic reduced dynamics, coefficient fitting, invariance checking, spectral-gap sweep, and rollout comparison. The backbone prediction Ω(r)=ω+c_i r² was observed as intercept 2.000000 and slope 0.350000 versus predicted 2.0 and 0.35; the radial law recovered λ=-0.080000 and c_r=-0.120000 versus the exact values. Transverse decay matched every swept γ from 0.04 to 0.64 within about 1e-9, and cubic SSM rollout RMSE was lower than the linear baseline at horizons 5/10/18 (0.0699/0.0890/0.0966 vs 0.3397/0.3833/0.3652).
- Confidence: 8/10
- Limitations: This is an analytically constructed two-dimensional normal-form system with an exactly known invariant graph, so the math predictions are strongly verified but neural encoder/decoder training, noisy data, forcing, higher-dimensional hidden states, model-selection of eigenvectors, GPU performance, and comparison against a trained GRU or neural ODE were not tested.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0313
- Paired wins: 2/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/132/bench_report.json)
- [bench_stage2.py](https://synthcore.org/code/132/bench_stage2.py)
- [report.md](https://synthcore.org/code/132/report.md)
- [report_bench_2026-08-30T155841.md](https://synthcore.org/code/132/report_bench_2026-08-30T155841.md)
- [results/metrics.json](https://synthcore.org/code/132/results/metrics.json)
- [ssm_experiment.py](https://synthcore.org/code/132/ssm_experiment.py)
- [Download all files as ZIP](https://synthcore.org/download/132)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.