# Spectral Template Continuation Layer

- ID: 756
- Canonical URL: https://synthcore.org/idea/756/spectral-template-continuation-layer
- API JSON: https://synthcore.org/api/idea/756.json
- API Markdown: https://synthcore.org/api/idea/756.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.11551](https://arxiv.org/abs/2607.11551)
- Category: dynamics
- Solves: accuracy, stability, memory
- ML areas: ssm, rnn, world-model, memory
- Math tags: dynamical-systems, linear-algebra, spectral-theory, control-theory, optimization
- Ratings: usefulness 8/10; difficulty 5/10; novelty 6/10

## Idea description

Add a non-autoregressive continuation layer to an RNN, SSM, or world model that predicts a future trajectory by solving for coefficients of a library of past trajectory windows and reusing those coefficients on the corresponding future windows. Unlike nearest-neighbor retrieval, the coefficients interpolate across multiple behaviors and can generalize to unseen systems whose output-visible eigenvalues are represented in the library.

## Mathematical statement

For each template trajectory, split its length-$T$ output vector into a past part and future part: $H_Y=[H_p;H_f]$, where $H_p\in\mathbb{R}^{rp\times N}$ contains the first $r$ time steps, $H_f\in\mathbb{R}^{(T-r)p\times N}$ contains the remaining steps, and $N$ is the number of templates. Given an observed past $y_p$, compute coefficients $g$ by $g=\arg\min_g\|H_pg-y_p\|_2^2+\lambda\|g\|_2^2$, then predict $\hat y_f=H_fg$. In the scalar-output linear case, let $s_{\mathrm{mix}}$ be the number of distinct output-visible eigenvalues across the library and let $s_{\mathrm{new}}$ be the number for a new diagonalizable system. If $r\geq s_{\mathrm{mix}}$, the library states are sufficiently rich, and $\spec_{C_{\mathrm{new}}}(A_{\mathrm{new}})\subseteq\bigcup_i\spec_{C_i}(A_i)$, then exact continuation is possible for noiseless data. Here $A_i$ and $C_i$ define prototype dynamics $x_{t+1}=A_ix_t$, $y_t=C_ix_t$, and $\spec_C(A)=\{\mu:C\ker(\mu I-A)\neq\{0\}\}$ is the output-visible spectrum. For a modal library $\Phi_T=[\lambda_j^t]_{t=0}^{T-1}$, partitioned as $\Phi_T=[\Phi_p;\Phi_f]$, the corresponding predictor is $\hat y_f=\Phi_f\Phi_p^\dagger y_p$.

## Key formulas

- $$g_\lambda=\arg\min_{g\in\mathbb{R}^{N}}\left\|H_pg-y_p\right\|_2^2+\lambda\left\|g\right\|_2^2=(H_p^{\mathsf T}H_p+\lambda I)^{-1}H_p^{\mathsf T}y_p,\qquad \hat y_f=H_fg_\lambda.$$
- $$\Phi_T=\begin{bmatrix}1&1&\cdots&1\\ \lambda_1&\lambda_2&\cdots&\lambda_s\\ \vdots&\vdots&&\vdots\\ \lambda_1^{T-1}&\lambda_2^{T-1}&\cdots&\lambda_s^{T-1}\end{bmatrix}=\begin{bmatrix}\Phi_p\\ \Phi_f\end{bmatrix},\qquad \hat y_f=\Phi_f\Phi_p^\dagger y_p.$$
- $$r\geq s_{\mathrm{mix}},\qquad \spec_{C_{\mathrm{new}}}(A_{\mathrm{new}})\subseteq\bigcup_{i=1}^{Q}\spec_{C_i}(A_i)\ \Longrightarrow\ \hat y_f=y_f\ \text{in the noiseless, sufficiently rich linear case}.$$
- $$\|\hat y_f-y_f\|_2\leq \|H_f\|_2\|H_p^\dagger\|_2\|\varepsilon_p\|_2,$$

## Implementation notes

1. Integration point: place this layer after an encoder has converted a context window into the same output space as the template library, or use it directly on normalized scalar or vector observations in an SSM or world-model predictor. Store a fixed matrix of templates with shape [N,T,p], and split it into H_p=[N,r,p] and H_f=[N,T-r,p]. 2. Pseudocode: normalize context y_p; flatten H_p and H_f; solve g=(H_p^T H_p+lambda I)^(-1)H_p^T y_p using Cholesky or QR; compute y_future=H_f g; optionally blend it with a learned residual predictor y_future+R_theta(context). Train the residual and encoder jointly, but initially freeze the library. Reject or downweight predictions when the condition number of H_p^T H_p is too large. 3. Computed quantities are the template matrices, ridge solve, singular values, and continuation. The exact visible spectrum is unavailable for arbitrary neural data, so estimate effective mode count by singular-value thresholding of H_p or by fitting a linear recurrence to each template. 4. First cheap experiment: generate length-64 trajectories from diagonal systems with 1--12 distinct eigenvalues in [-0.95,0.95], hold out systems with eigenvalues inside the training-library spectral union, and compare this layer with a GRU, linear AR model, and nearest-template retrieval. Sweep context length r. The paper predicts a sharp transition: normalized forecast error should drop substantially once r reaches the effective number s of visible modes, while error and coefficient variance should rise near singular or uncovered spectra. Measure the empirical transition r_star and test whether r_star agrees with the estimated s within 20%; also test whether ridge regularization prevents noise-induced blowup according to the bound's dependence on the smallest singular value of H_p.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built a NumPy spectral-template continuation layer using pseudoinverse or ridge coefficients, with nearest-template and AR baselines. The predicted mode threshold s_mix=7 was observed exactly at r=7; forecast perturbation error scaled with noise amplitude and stayed below the stated ||H_f||·||H_p^dagger||·||epsilon|| bound at all tested amplitudes. Ridge reduced coefficient variance by about 18.6x at lambda=1e-3, but introduced substantial bias, so this is a stability/variance win rather than an unconditional accuracy win.

### Mechanism check

- Verdict: Built a NumPy spectral-template continuation layer using pseudoinverse or ridge coefficients, with nearest-template and AR baselines. The predicted mode threshold s_mix=7 was observed exactly at r=7; forecast perturbation error scaled with noise amplitude and stayed below the stated ||H_f||·||H_p^dagger||·||epsilon|| bound at all tested amplitudes. Ridge reduced coefficient variance by about 18.6x at lambda=1e-3, but introduced substantial bias, so this is a stability/variance win rather than an unconditional accuracy win.
- Confidence: 9/10
- Limitations: Only synthetic scalar diagonal/modal trajectories were tested. The continuation target was generated from the library's represented modes, so this verifies the stated in-span spectral mechanism but not genuinely unseen systems, vector outputs, learned encoders, GRUs, SSMs, FLOPs, runtime, template storage, or a trained residual predictor.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/144/bench_report.json)
- [experiment.py](https://synthcore.org/code/144/experiment.py)
- [report.md](https://synthcore.org/code/144/report.md)
- [report_bench_2026-08-30T174357.md](https://synthcore.org/code/144/report_bench_2026-08-30T174357.md)
- [results.json](https://synthcore.org/code/144/results.json)
- [spectral_bench.py](https://synthcore.org/code/144/spectral_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/144)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.