✗ Failed on benchmark 2026

Spectral Template Continuation Layer

Implementation & benchmark of arXiv:2607.11551 — Machines that Predict Trajectories from Templates

Usefulness8/10

Difficulty5/10

Novelty6/10

Dynamics Accuracy Stability Memory Ssm Rnn World model Memory Dynamical systems Linear algebra Spectral theory Control theory Optimization

Source paper: Machines that Predict Trajectories from Templates arXiv:2607.11551 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Add a non-autoregressive continuation layer to an RNN, SSM, or world model that predicts a future trajectory by solving for coefficients of a library of past trajectory windows and reusing those coefficients on the corresponding future windows. Unlike nearest-neighbor retrieval, the coefficients interpolate across multiple behaviors and can generalize to unseen systems whose output-visible eigenvalues are represented in the library.

Formulas

$$g_\lambda=\arg\min_{g\in\mathbb{R}^{N}}\left\|H_pg-y_p\right\|_2^2+\lambda\left\|g\right\|_2^2=(H_p^{\mathsf T}H_p+\lambda I)^{-1}H_p^{\mathsf T}y_p,\qquad \hat y_f=H_fg_\lambda.$$

$$\Phi_T=\begin{bmatrix}1&1&\cdots&1\\ \lambda_1&\lambda_2&\cdots&\lambda_s\\ \vdots&\vdots&&\vdots\\ \lambda_1^{T-1}&\lambda_2^{T-1}&\cdots&\lambda_s^{T-1}\end{bmatrix}=\begin{bmatrix}\Phi_p\\ \Phi_f\end{bmatrix},\qquad \hat y_f=\Phi_f\Phi_p^\dagger y_p.$$

$$r\geq s_{\mathrm{mix}},\qquad \spec_{C_{\mathrm{new}}}(A_{\mathrm{new}})\subseteq\bigcup_{i=1}^{Q}\spec_{C_i}(A_i)\ \Longrightarrow\ \hat y_f=y_f\ \text{in the noiseless, sufficiently rich linear case}.$$

$$\|\hat y_f-y_f\|_2\leq \|H_f\|_2\|H_p^\dagger\|_2\|\varepsilon_p\|_2,$$

Mathematical statement

For each template trajectory, split its length-$T$ output vector into a past part and future part: $H_Y=[H_p;H_f]$, where $H_p\in\mathbb{R}^{rp\times N}$ contains the first $r$ time steps, $H_f\in\mathbb{R}^{(T-r)p\times N}$ contains the remaining steps, and $N$ is the number of templates. Given an observed past $y_p$, compute coefficients $g$ by $g=\arg\min_g\|H_pg-y_p\|_2^2+\lambda\|g\|_2^2$, then predict $\hat y_f=H_fg$. In the scalar-output linear case, let $s_{\mathrm{mix}}$ be the number of distinct output-visible eigenvalues across the library and let $s_{\mathrm{new}}$ be the number for a new diagonalizable system. If $r\geq s_{\mathrm{mix}}$, the library states are sufficiently rich, and $\spec_{C_{\mathrm{new}}}(A_{\mathrm{new}})\subseteq\bigcup_i\spec_{C_i}(A_i)$, then exact continuation is possible for noiseless data. Here $A_i$ and $C_i$ define prototype dynamics $x_{t+1}=A_ix_t$, $y_t=C_ix_t$, and $\spec_C(A)=\{\mu:C\ker(\mu I-A)\neq\{0\}\}$ is the output-visible spectrum. For a modal library $\Phi_T=[\lambda_j^t]_{t=0}^{T-1}$, partitioned as $\Phi_T=[\Phi_p;\Phi_f]$, the corresponding predictor is $\hat y_f=\Phi_f\Phi_p^\dagger y_p$.

Implementation notes

1. Integration point: place this layer after an encoder has converted a context window into the same output space as the template library, or use it directly on normalized scalar or vector observations in an SSM or world-model predictor. Store a fixed matrix of templates with shape [N,T,p], and split it into H_p=[N,r,p] and H_f=[N,T-r,p]. 2. Pseudocode: normalize context y_p; flatten H_p and H_f; solve g=(H_p^T H_p+lambda I)^(-1)H_p^T y_p using Cholesky or QR; compute y_future=H_f g; optionally blend it with a learned residual predictor y_future+R_theta(context). Train the residual and encoder jointly, but initially freeze the library. Reject or downweight predictions when the condition number of H_p^T H_p is too large. 3. Computed quantities are the template matrices, ridge solve, singular values, and continuation. The exact visible spectrum is unavailable for arbitrary neural data, so estimate effective mode count by singular-value thresholding of H_p or by fitting a linear recurrence to each template. 4. First cheap experiment: generate length-64 trajectories from diagonal systems with 1--12 distinct eigenvalues in [-0.95,0.95], hold out systems with eigenvalues inside the training-library spectral union, and compare this layer with a GRU, linear AR model, and nearest-template retrieval. Sweep context length r. The paper predicts a sharp transition: normalized forecast error should drop substantially once r reaches the effective number s of visible modes, while error and coefficient variance should rise near singular or uncovered spectra. Measure the empirical transition r_star and test whether r_star agrees with the estimated s within 20%; also test whether ridge regularization prevents noise-induced blowup according to the bound's dependence on the smallest singular value of H_p.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a NumPy spectral-template continuation layer using pseudoinverse or ridge coefficients, with nearest-template and AR baselines. The predicted mode threshold s_mix=7 was observed exactly at r=7; forecast perturbation error scaled with noise amplitude and stayed below the stated ||H_f||·||H_p^dagger||·||epsilon|| bound at all tested amplitudes. Ridge reduced coefficient variance by about 18.6x at lambda=1e-3, but introduced substantial bias, so this is a stability/variance win rather than an unconditional accuracy win.

Agent confidence: 9/10

Baseline: Nearest-template relative forecast error 0.9933; AR(5) relative error 0.0851.
Idea: Modal continuation relative error 4.6e-15 noiseless at r=10; first exact continuation at r=7 versus predicted s_mix=7. At r=8 with sigma=2e-4 observation noise, unregularized mean relative error was 0.00204 and lambda=1e-3 was 0.19966, while coefficient variance fell from 2.513e-4 to 1.348e-5. Noise-sweep forecast errors were 8.54e-7 to 6.86e-4 for noise norms 1e-6 to 1e-3, with every bound check passing.

Limitations: Only synthetic scalar diagonal/modal trajectories were tested. The continuation target was generated from the library's represented modes, so this verifies the stated in-span spectral mechanism but not genuinely unseen systems, vector outputs, learned encoders, GRUs, SSMs, FLOPs, runtime, template storage, or a trained residual predictor.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.11551) Sep 2, 2026

Significantly WORSE than the tuned baseline (+189.3%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Sequence modeling
Model: transformer_tiny
Paired seeds: 8
Baseline mean: 0.3913
Idea mean: 1.1322
Effect (Δ): 0.7408 (+189.3%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±115.5%
Mechanism: Not confirmed ✗
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

Only the sequence benchmark track was tested. Multi-step rollout, dynamics, alternative template construction, context-length sweeps, learned residual blending, and larger template libraries were not evaluated. The frozen library was sampled from training windows and the learned encoder mapped contexts into template-past space; this tests neural transfer rather than the noiseless modal theorem directly.

How to run: python3 spectral_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 30, 2026 latest Failed ✗

Agent confidence: 10/10

Implemented an end-to-end spectral template continuation head on the structurally matched sequence forecasting track, using the same transformer backbone, training budget, paired seeds, and shared learning-rate sweep as the direct baseline. The idea was significantly worse: baseline MSE 0.391339 versus idea MSE 1.132171, paired delta +0.740832, permutation p=0.0081, with 0/8 paired wins. The trained-model mechanism signature was not confirmed because continuation did not reduce held-out forecast error.

Baseline: sequence/transformer_tiny, best lr=0.001; 8-seed MSE mean 0.391339, std 0.034275; sweep means: lr=0.001 -> 0.379792, lr=0.003 -> 0.435053, lr=0.01 -> 0.907954
Idea: spectral continuation with ridge=1e-5, best lr=0.001; 8-seed MSE mean 1.132171, std 0.526083; sweep means: lr=0.001 -> 0.855925, lr=0.003 -> 1.743744, lr=0.01 -> 6.425328
Paired comparison: delta_mean=+0.740832 (idea minus baseline; lower is better), p_value=0.0081, idea_wins=0/8, verdict=idea worse (significant)

Benchmark result