# DMDc-Initialized Marginal-Stable Neural SSM

- ID: 552
- Canonical URL: https://synthcore.org/idea/552/dmdc-initialized-marginal-stable-neural-ssm
- API JSON: https://synthcore.org/api/idea/552.json
- API Markdown: https://synthcore.org/api/idea/552.md
- Verification status: mechanism_failed
- Source: [arXiv:2607.07594](https://arxiv.org/abs/2607.07594)
- Category: architecture
- Solves: stability, sample-efficiency, accuracy
- ML areas: ssm, rnn, world-model, initialization
- Math tags: dynamical-systems, linear-algebra, spectral-theory, numerical-analysis
- Ratings: usefulness 7/10; difficulty 5/10; novelty 6/10

## Idea description

Replace the unconstrained transition of a recurrent or state-space neural network with a DMDc-initialized linear latent transition plus a learned nonlinear residual. Estimate the transition from a short warm-up dataset using Hankel delay coordinates, retain eigenmodes with decay rates near the unit circle for long-term memory, and let the neural residual model dynamics not explained by the linear backbone. This should make long-horizon prediction and slowly varying signals easier to learn while reducing unstable recurrent training.

## Mathematical statement

The paper models the observed voltage as a forced nonlinear system, $\frac{dV_t}{dt}=f(V_t,I)$, where $V_t$ is the measured output, $I$ is an external input, and $f$ is an unknown nonlinear vector field. Its constructive lifting uses delayed observations; for a scalar output $y_t$ and delay length $L$, define the Hankel state $z_t=[y_t,y_{t-1},\ldots,y_{t-L+1}]^\top$. The adapted controlled-DMD relation is $z_{t+1}\approx A z_t+B u_t$, where $u_t$ is the input, $A$ is the learned autonomous transition, and $B$ is input coupling. Estimate both matrices by least squares: $[A\ B]=Z_+[Z_-;U_-]^\dagger$, where $Z_-=[z_0,\ldots,z_{T-1}]$, $Z_+=[z_1,\ldots,z_T]$, $U_-=[u_0,\ldots,u_{T-1}]$, and $\dagger$ denotes the Moore-Penrose pseudoinverse. Spectral analysis computes $Aw_j=\lambda_jw_j$; modes with $|\lambda_j|$ close to one are slowly decaying or marginally stable. The neural model uses $h_{t+1}=A h_t+B u_t+g_\theta(h_t,u_t)$, with $g_\theta$ a small residual network. To prevent exploding dynamics, optionally project eigenvalues after each optimizer step using $\lambda_j\leftarrow \lambda_j/\max(1,|\lambda_j|)$, while preserving a selected slow-mode band $|\lambda_j|\in[1-\epsilon,1]$. Here $h_t$ is the learned latent state, unlike the paper's battery-specific observable $z_t$.

## Key formulas

- $$\frac{dV_t}{dt}=f(V_t,I)$$
- $$z_t=[y_t,y_{t-1},\ldots,y_{t-L+1}]^\top$$
- $$[A\;B]=Z_+\begin{bmatrix}Z_-\\U_-\end{bmatrix}^{\dagger},\qquad z_{t+1}\approx Az_t+Bu_t$$
- $$h_{t+1}=Ah_t+Bu_t+g_\theta(h_t,u_t),\qquad Aw_j=\lambda_jw_j$$

## Implementation notes

Integrate this at the transition step of a compact recurrent model or diagonal/state-space sequence model. First choose a scalar or low-dimensional observed signal y_t, an input vector u_t, delay length L, and latent dimension d. Build causal delay vectors z_t by stacking the most recent L observations; if the raw observation is multidimensional, concatenate delayed channels and reduce the resulting matrix to d dimensions with truncated SVD. Construct Z_- and Z_+ from adjacent delay states and U_- from aligned inputs. Compute the regularized DMDc estimate [A B]=Z_+[Z_-;U_-]^T(([Z_-;U_-][Z_-;U_-]^T)+alpha I)^{-1}, with small alpha for noisy data. Take the rank-d SVD truncation before fitting if the Hankel matrix is large. Initialize the neural latent state h_0 from the projected first delay vector, then run: h_pred=A h_t+B u_t; r=g_theta(h_t,u_t); h_{t+1}=h_pred+r; yhat_t=readout(h_t). Start with g_theta as a two-layer width-2d MLP and optionally scale its output by a learnable gate initialized to 0. Compute eigendecomposition of A, record eigenvalues, and identify slow modes by minimizing ||lambda|-1| among stable eigenvalues. During training compare three variants: random A, DMDc A without constraints, and DMDc A with spectral projection or a penalty alpha_s sum_j max(0,|lambda_j|-1)^2. What comes directly from the mathematics is the delay lifting, least-squares DMDc initialization, eigenvalue-based mode selection, and stability projection; the residual network, rank choice, and penalty strength are empirical design choices. The first cheap experiment should use a small copy task, adding-machine task, or Lorenz-system forecasting with sparse inputs, followed by a real electricity or battery time-series benchmark. Train all variants at equal parameter count and FLOPs. Measure validation loss versus optimization steps, gradient norm variance, spectral radius, and mean-squared error at horizons 10x and 50x the training rollout. Success means faster early loss descent, no exploding trajectories, and lower long-horizon error than the randomly initialized recurrent baseline.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a compact DMDc-initialized residual latent model with Hankel delay lifting, ridge DMDc fitting, eigenvalue inspection, and a random-transition recurrent baseline. The numerical checks confirmed exact noiseless recovery, ridge error scaling linearly with regularization (log-log slope 0.9999), and bounded projected dynamics, but the forecasting experiment showed no win: DMDc had higher validation, horizon-10, and horizon-50 MSE than the random baseline.

### Mechanism check

- Verdict: Built a compact DMDc-initialized residual latent model with Hankel delay lifting, ridge DMDc fitting, eigenvalue inspection, and a random-transition recurrent baseline. The numerical checks confirmed exact noiseless recovery, ridge error scaling linearly with regularization (log-log slope 0.9999), and bounded projected dynamics, but the forecasting experiment showed no win: DMDc had higher validation, horizon-10, and horizon-50 MSE than the random baseline.
- Confidence: 9/10
- Limitations: Only a small synthetic scalar nonlinear forecasting task was tested; no electricity or battery benchmark, FLOP-matched parameter audit, gradient-variance analysis, multiple-seed study, or implemented post-update spectral projection was evaluated. The DMDc backbone was frozen in this experiment, and its noisy estimate was slightly unstable, which may disadvantage the idea.

## Artifacts

- [experiment.py](https://synthcore.org/code/254/experiment.py)
- [report.md](https://synthcore.org/code/254/report.md)
- [results.json](https://synthcore.org/code/254/results.json)
- [Download all files as ZIP](https://synthcore.org/download/254)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.