# Spectral Burn-In and Retrieval Switch

- ID: 937
- Canonical URL: https://synthcore.org/idea/937/spectral-burn-in-and-retrieval-switch
- API JSON: https://synthcore.org/api/idea/937.json
- API Markdown: https://synthcore.org/api/idea/937.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.14885](https://arxiv.org/abs/2607.14885)
- Category: dynamics
- Solves: speedup, stability, accuracy
- ML areas: rnn, ssm, world-model, retrieval, inference-speedup
- Math tags: dynamical-systems, control-theory, spectral-theory, statistics
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Use the observer contraction rate as an online inference controller. Run the latent observer when its estimated contraction is strong, and invoke expensive retrieval or latent-state reinitialization only when contraction is weak or observation residuals indicate model mismatch.

## Mathematical statement

Let \(r_t\) denote the local contraction estimate, equal to \(\rho(A_{cl,t})\) for a linear observer or an estimated Jacobian norm for a nonlinear model. Under contraction, state error follows \(E_k\approx E_0r_t^k\), so the predicted burn-in to reach \(\varepsilon\) is \(k_t=\lceil\log(\varepsilon/E_0)/\log r_t\rceil\) for \(0<r_t<1\). The paper contrasts this with recurrence cost \(\mathbb{E}[T_{rec}(\varepsilon)]\asymp\varepsilon^{-d}\), where \(d\) is pointwise attractor dimension. The controller uses observer rollout if \(r_t\le r_{max}<1\) and residuals are small; otherwise it retrieves a similar context or resets the latent state.

## Key formulas

- $$E_k\approx E_0r^k,\qquad k_\varepsilon=\left\lceil\frac{\log(\varepsilon/E_0)}{\log r}\right\rceil,\quad 0<r<1.$$
- $$\mathbb{E}[T_{rec}(\varepsilon)]\asymp \varepsilon^{-d},\qquad \mathbb{E}[T_{obs}(\varepsilon)]\asymp \frac{\log(1/\varepsilon)}{1-r^2}.$$
- $$\text{observer mode}\iff r_t\le r_{max}\ \wedge\ \|y_t-g(z_t)\|\le\tau;\qquad\text{otherwise retrieve or reset}.$$

## Implementation notes

Attach the controller to a recurrent state-space model or a transformer with recurrent latent memory. At each inference step compute the observation residual \(q_t=\|y_t-g(z_t)\|\), estimate \(r_t\) using 3--10 power iterations on Jacobian-vector products, and maintain an error estimate \(\hat E_t\). Set \(k_t=\lceil\log(\varepsilon/E_0)/\log(\max(r_t,r_{min}))\rceil\), where \(E_0\) is calibrated from validation transients and \(r_{min}\) avoids numerical singularities. Continue cheap observer rollout when \(r_t\le r_{max}\) and \(q_t\le\tau\). Otherwise retrieve a nearest latent delay window from a memory bank, blend it with the current state, or rerun the encoder. The recurrence and observer scaling laws are taken from the paper; \(r_t\), \(E_0\), \(\tau\), and \(d\) are estimated empirically. First test on Lorenz-63 with noisy scalar observations, comparing always-retrieve, always-observer, and adaptive switching at matched prediction error. Sweep \(\varepsilon\) from \(10^{-2}\) to \(10^{-6}\). The predicted signature is observer burn-in linear in \(\log(1/\varepsilon)\), retrieval frequency falling sharply for \(r_t<1\), and a compute crossover near \(\varepsilon^{-d}=\log(1/\varepsilon)/(1-r_t^2)\).

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built and ran a reproducible scalar contraction verification plus a noisy Lorenz-63 observer/retrieval switch in run_experiment.py. The core mechanism manifested: predicted burn-in exactly matched observed recurrence steps, log(1/epsilon) slopes agreed with 1/|log r| within 2.2%, and the stability boundary occurred at r=1. Adaptive switching detected 79.4% of injected mismatch steps while reducing retrieval rate to 35.9%, but it did not improve accuracy in this toy implementation: adaptive RMSE was 5.09 versus 0.75 for always-observer and 14.40 for always-retrieve.

### Mechanism check

- Verdict: Built and ran a reproducible scalar contraction verification plus a noisy Lorenz-63 observer/retrieval switch in run_experiment.py. The core mechanism manifested: predicted burn-in exactly matched observed recurrence steps, log(1/epsilon) slopes agreed with 1/|log r| within 2.2%, and the stability boundary occurred at r=1. Adaptive switching detected 79.4% of injected mismatch steps while reducing retrieval rate to 35.9%, but it did not improve accuracy in this toy implementation: adaptive RMSE was 5.09 versus 0.75 for always-observer and 14.40 for always-retrieve.
- Confidence: 8/10
- Limitations: The Lorenz observer uses a hand-designed fixed contraction proxy rather than 3-10 Jacobian-vector power iterations, and retrieval uses nearest scalar observation rather than a learned latent delay-window index. No trained neural state-space model, real retrieval system, FLOP profiling, epsilon sweep in the learned model, or repeated random seeds was tested. The adaptive policy is evidence for the switching mechanism and math, not a demonstrated end-to-end accuracy or speed win.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/396/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/396/bench_report.json)
- [report.md](https://synthcore.org/code/396/report.md)
- [report_bench_2026-08-31T110915.md](https://synthcore.org/code/396/report_bench_2026-08-31T110915.md)
- [results.json](https://synthcore.org/code/396/results.json)
- [run_experiment.py](https://synthcore.org/code/396/run_experiment.py)
- [Download all files as ZIP](https://synthcore.org/download/396)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.