# Recursive Noise-Corrected Latent Dynamics

- ID: 1048
- Canonical URL: https://synthcore.org/idea/1048/recursive-noise-corrected-latent-dynamics
- API JSON: https://synthcore.org/api/idea/1048.json
- API Markdown: https://synthcore.org/api/idea/1048.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.17065](https://arxiv.org/abs/2607.17065)
- Category: dynamics
- Solves: stability, accuracy, sample-efficiency
- ML areas: ssm, world-model, training-dynamics, optimizer
- Math tags: dynamical-systems, linear-algebra, statistics, spectral-theory, optimization
- Ratings: usefulness 8/10; difficulty 6/10; novelty 6/10

## Idea description

Insert an online errors-in-variables subspace estimator into a latent state-space neural network. A fixed recent window of encoder features and controls is used to estimate a noise-corrected low-dimensional state subspace and refit the latent transition and readout matrices, allowing the model to follow sensor degradation or changing operating conditions without replaying the entire dataset.

## Mathematical statement

Let $o_k\in\mathbb{R}^{p}$ be an observed feature vector, $u_k\in\mathbb{R}^{m}$ an input, and $o_k=z_k+\varepsilon_k$ with unknown diagonal measurement-noise covariance $R_k=\operatorname{diag}(\sigma_{1,k}^2,\ldots,\sigma_{p,k}^2)$. Maintain exponentially weighted covariance statistics with forgetting factor $\gamma\in(0,1]$: $C_k=\gamma C_{k-1}+(1-\gamma)o_ko_k^\top$ and $\widehat R_k=\gamma\widehat R_{k-1}+(1-\gamma)\operatorname{diag}(r_kr_k^\top)$, where $r_k=o_k-\widehat o_k$ is the decoder residual. Form the corrected covariance $S_k=C_k-\widehat R_k$, symmetrize it, and retain eigenvectors whose eigenvalues exceed a noise floor. If $U_k\in\mathbb{R}^{p\times r_k}$ contains the retained eigenvectors, define latent coordinates $x_k=U_k^\top o_k$. Fit $x_{k+1}=A_kx_k+B_ku_k+\xi_k$ by weighted ridge regression, and project $A_k$ to spectral radius below $1-\epsilon$ for stable free-running prediction. The essential transferable mechanism is recursive low-rank covariance tracking after subtracting heteroskedastic observation noise.

## Key formulas

- $$C_k=\gamma C_{k-1}+(1-\gamma)o_ko_k^\top,\qquad \widehat R_k=\gamma\widehat R_{k-1}+(1-\gamma)\operatorname{diag}(r_kr_k^\top),$$
- $$S_k=\frac{(C_k-\widehat R_k)+(C_k-\widehat R_k)^\top}{2},\qquad S_kU_k=U_k\Lambda_k,\qquad r_k=\#\{i:\lambda_{i,k}>\tau_{i,k}\},\qquad x_k=U_k^\top o_k,$$
- $$[\widehat A_k\ \widehat B_k]=\arg\min_{A,B}\sum_{j=k-L+1}^{k-1}\gamma^{k-1-j}\left\|x_{j+1}-Ax_j-Bu_j\right\|_2^2+\lambda\left(\|A\|_F^2+\|B\|_F^2\right),$$
- $$A_k\leftarrow(1-\epsilon)\frac{A_k}{\max\left(1,\rho(A_k)/(1-\epsilon)\right)},\qquad \rho(A_k)=\max_i|\operatorname{eig}_i(A_k)|.$$

## Implementation notes

1. Exact integration point: use a small neural encoder $e_\theta$ to map raw observations to $o_k$, and a decoder or prediction head to reconstruct observations. Every $L$ steps, or once per minibatch in streaming training, run the recursive estimator on the last $L$ encoded features and controls. Use the resulting $U_k$ as a stop-gradient coordinate transform, and train a latent SSM predictor with $x_{k+1}=A_kx_k+B_ku_k$; alternatively initialize an SSM transition from $(A_k,B_k)$ and continue gradient training. 2. Pseudocode: compute $o=e_\theta(y)$ and residual $r=y-\operatorname{decode}(o)$; update $C\leftarrow\gamma C+(1-\gamma)oo^\top$ and $R\leftarrow\gamma R+(1-\gamma)\operatorname{diag}(rr^\top)$; eigendecompose $S=C-R$; retain eigenvectors with $\lambda_i>\tau_i=cR_{ii}$ or a validation-calibrated threshold; project $x=U^\top o$; solve the weighted ridge regression for $A,B$; project $A$ to $\rho(A)\leq1-\epsilon$; train the encoder and decoder using reconstruction plus latent one-step and multistep prediction loss. 3. Computed from the paper's mechanism: recursive covariance, fixed-lag statistics, noise-corrected PCA, and recursive model refitting. Estimated empirically: residual-based $R$, threshold constant $c$, forgetting factor $\gamma$, and ridge coefficient $\lambda$. 4. First cheap experiment: use a noisy Lorenz-63 or switched linear dynamical system, encode observations with a two-layer MLP, and compare a standard neural SSM, online PCA initialization, and this method. Inject a sensor variance jump at time $T$ and a transition-matrix change at $2T$. The quantitative prediction is that corrected PCA will suppress dimensions whose eigenvalues are below the estimated noise floor, while the uncorrected baseline will increase rank after the variance jump. For a scalar parameter jump, fitted $A_k$ should adapt exponentially with characteristic time approximately $1/(1-\gamma)$ samples; measure this against the predicted lag within 20%. Stable rollout divergence should begin only when unconstrained estimated $\rho(A_k)$ exceeds one, and the spectral projection should prevent that boundary.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built a NumPy MVP for recursive noise-corrected covariance tracking, latent subspace extraction, weighted ridge transition refitting, and spectral-radius projection. The three mechanism-level checks matched predictions: corrected top eigenvalues stayed near 4.0 as noise variance increased, EWMA 63% response steps matched -1/log(gamma) within discrete-rounding error, and all projected radii were below 0.95. In the switched linear toy, corrected dynamics achieved MSE 0.02544 versus 0.02711 for raw PCA, but the fitted transitions were already stable, so no empirical instability-prevention win was observed.

### Mechanism check

- Verdict: Built a NumPy MVP for recursive noise-corrected covariance tracking, latent subspace extraction, weighted ridge transition refitting, and spectral-radius projection. The three mechanism-level checks matched predictions: corrected top eigenvalues stayed near 4.0 as noise variance increased, EWMA 63% response steps matched -1/log(gamma) within discrete-rounding error, and all projected radii were below 0.95. In the switched linear toy, corrected dynamics achieved MSE 0.02544 versus 0.02711 for raw PCA, but the fitted transitions were already stable, so no empirical instability-prevention win was observed.
- Confidence: 8/10
- Limitations: The decoder residual stream used the known simulated sensor residual rather than a trained neural encoder/decoder residual, so this validates the covariance mechanism rather than full end-to-end neural SSM training. Only one deterministic switched-linear toy stream, one window/ridge configuration, and a fixed retained rank for regression were tested; no Lorenz system, learned encoder, compute-speed comparison, or multi-seed statistical test was run.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/187/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/187/bench_report.json)
- [experiment.py](https://synthcore.org/code/187/experiment.py)
- [report.md](https://synthcore.org/code/187/report.md)
- [report_bench_2026-08-30T194623.md](https://synthcore.org/code/187/report_bench_2026-08-30T194623.md)
- [results.json](https://synthcore.org/code/187/results.json)
- [Download all files as ZIP](https://synthcore.org/download/187)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.