# Annealed-resonance recurrent dynamics

- ID: 1854
- Canonical URL: https://synthcore.org/idea/1854/annealed-resonance-recurrent-dynamics
- API JSON: https://synthcore.org/api/idea/1854.json
- API Markdown: https://synthcore.org/api/idea/1854.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.05649](https://arxiv.org/abs/2608.05649)
- Category: dynamics
- Solves: stability, accuracy, memory
- ML areas: rnn, ssm, optimizer, training-dynamics
- Math tags: dynamical-systems, spectral-theory, stochastic-processes, statistics
- Ratings: usefulness 8/10; difficulty 6/10; novelty 7/10

## Idea description

Replace a deterministic recurrent transition by an iid-random family of transitions and explicitly control the spectrum of the corresponding annealed Koopman operator. Nontrivial eigenvalues inside the unit disk give a measurable exponential memory-decay envelope, while complex eigenvalues provide stable oscillatory memory modes useful for long-horizon sequence prediction.

## Mathematical statement

Let $\xi_t$ be iid random variables and let the hidden-state transition be $h_{t+1}=T_{\xi_t}(h_t)$. For an observable $\varphi$ on hidden states, define the annealed Koopman operator $(K\varphi)(h)=\mathbb{E}_{\xi}[\varphi(T_{\xi}(h))]$. Independence implies $\mathbb{E}[\varphi(h_n)\mid h_0]=K^n\varphi(h_0)$. If the centered operator $K_0$ has eigenvalues, called annealed resonances, $\lambda_j$ with $|\lambda_j|<1$, then correlations have the asymptotic form $C_n\approx\sum_j c_j\lambda_j^n$. Their envelope decays approximately as $\rho^n$, where $\rho=\max_j|\lambda_j|$; complex $\lambda_j=\rho e^{i\theta}$ produce decay rate $\rho$ and oscillation frequency $\theta$. The constant invariant feature with eigenvalue $1$ must be removed before imposing the stability constraint.

## Key formulas

- $$(K\varphi)(h)=\mathbb{E}_{\xi}\left[\varphi\left(T_{\xi}(h)\right)\right],\qquad \mathbb{E}[\varphi(h_n)\mid h_0]=K^n\varphi(h_0).$$
- $$C_n(f,g)=\langle f,K^n g\rangle_\mu-\langle f,1\rangle_\mu\langle 1,g\rangle_\mu\approx\sum_j c_j\lambda_j^n,$$
- $$\rho(K_0)=\max_j|\lambda_j|<1\quad\Longrightarrow\quad |C_n|\lesssim C\rho(K_0)^n,$$

## Implementation notes

1. Integration point: use a stochastic RNN or state-space model with $M$ transition branches, $h_{t+1}=T_{\xi_t,\theta}(h_t)$, where $\xi_t$ is sampled independently at each sequence step or per training example. Start with affine branches $T_m(h)=\tanh(A_mh+B_mx+b_m)$, and first perform the diagnostic with linear recurrent branches. 2. Pseudocode: roll out independent branch samples, collect feature vectors $z_t=\phi(h_t)$, and fit an empirical annealed Koopman matrix by ridge regression, $\widehat K=\arg\min_K\sum_t\|z_{t+1}-Kz_t\|^2+\alpha\|K\|_F^2$. Include a constant feature, estimate its invariant mode, then project that mode out. Compute eigenvalues $\widehat\lambda_j$ and add $L_{\mathrm{res}}=\gamma[\max(0,\max_j|\widehat\lambda_j|-\rho_*)]^2$. For oscillatory memory, add a penalty matching selected poles $\rho_*e^{\pm i\theta_*}$. 3. Computed from the paper: the averaged-operator construction, iid composition rule $K^n$, and resonance-to-correlation correspondence. Estimated empirically: the finite-feature Koopman matrix, invariant distribution, eigenvalues, and correlation coefficients $c_j$. 4. First cheap experiment: train a 32-unit stochastic RNN on adding and delayed-copy tasks, comparing a shared deterministic RNN, unconstrained random branches, and resonance-constrained branches. Measure hidden-state autocorrelations and test loss versus lag. The falsifiable prediction is that, after removing the constant mode, correlation envelopes follow $\rho^n$ and have half-life $n_{1/2}=\log(1/2)/\log\rho$ within about 20%. Instability or growing correlations should appear when the estimated spectral radius crosses $1$, while targeted complex poles should produce a spectral peak near frequency $\theta_*/(2\pi)$.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: no_effect
- Verdict: Built an annealed linear recurrent-dynamics MVP with iid transition branches, exact branch averaging, Monte Carlo correlations, spectral/oscillation diagnostics, and a small delayed-copy readout comparison. The iid identity matched K^n with absolute sampling error 0.00194; predicted half-lives over rho=0.60–0.92 matched interpolated measurements within 4.7%; and the rho<1 versus rho>1 sweep tracked the predicted per-step amplitude ratios to about 1e-4. Complex-pole frequency estimates were close to theta/(2pi), but the annealed branches did not beat the deterministic baseline on delayed-copy MSE (0.9100 vs 0.9008), so this is mechanism confirmation rather than an observed task-level win.

### Mechanism check

- Verdict: Built an annealed linear recurrent-dynamics MVP with iid transition branches, exact branch averaging, Monte Carlo correlations, spectral/oscillation diagnostics, and a small delayed-copy readout comparison. The iid identity matched K^n with absolute sampling error 0.00194; predicted half-lives over rho=0.60–0.92 matched interpolated measurements within 4.7%; and the rho<1 versus rho>1 sweep tracked the predicted per-step amplitude ratios to about 1e-4. Complex-pole frequency estimates were close to theta/(2pi), but the annealed branches did not beat the deterministic baseline on delayed-copy MSE (0.9100 vs 0.9008), so this is mechanism confirmation rather than an observed task-level win.
- Confidence: 8/10
- Limitations: This is a 2D linear toy system with a ridge-free exact constructed average rather than a trained nonlinear stochastic RNN. It does not test resonance regularization, learned branch parameters, adding tasks, long-horizon nonlinear prediction, invariant-feature projection, or compute/FLOP-normalized training. The oscillation diagnostic uses a short finite FFT sweep and has visible frequency-bin quantization.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.17715
- Paired wins: 3/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_report.json](https://synthcore.org/code/658/bench_report.json)
- [bench_run.py](https://synthcore.org/code/658/bench_run.py)
- [experiment.py](https://synthcore.org/code/658/experiment.py)
- [report.md](https://synthcore.org/code/658/report.md)
- [report_bench_2026-08-31T210803.md](https://synthcore.org/code/658/report_bench_2026-08-31T210803.md)
- [results.json](https://synthcore.org/code/658/results.json)
- [Download all files as ZIP](https://synthcore.org/download/658)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.