# Resonance-Aware Stochastic RNN Control

- ID: 1120
- Canonical URL: https://synthcore.org/idea/1120/resonance-aware-stochastic-rnn-control
- API JSON: https://synthcore.org/api/idea/1120.json
- API Markdown: https://synthcore.org/api/idea/1120.md
- Verification status: beats_baseline
- Source: [arXiv:2607.18771](https://arxiv.org/abs/2607.18771)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: rnn, ssm, training-dynamics, regularization
- Math tags: dynamical-systems, spectral-theory, stochastic-processes, harmonic-analysis, linear-algebra
- Ratings: usefulness 7/10; difficulty 6/10; novelty 7/10

## Idea description

Estimate the leading complex resonances of the noise-averaged hidden-state dynamics of a stochastic RNN and use them to detect or control statistically persistent oscillations. The key design principle is to treat resonance radius and Lyapunov growth as independent signals: hidden trajectories can be Lyapunov-stable while the annealed dynamics still produce narrow-band ringing because a transfer-operator eigenvalue lies close to the unit circle.

## Mathematical statement

Let the stochastic hidden update be h_(t+1)=F_theta(h_t,x_t,xi_t), where xi_t is injected noise and K_theta(h,A)=Pr(F_theta(h,x_t,xi_t) belongs to A) is the annealed Markov kernel after averaging over noise and optionally inputs. Its transfer operator is (P_theta f)(h)=E[f(F_theta(h,x_t,xi_t))]. A Ruelle–Pollicott resonance is an eigenvalue z_j=r_j exp(i omega_j) satisfying P_theta phi_j=z_j phi_j, with 0<=r_j<1. For an observable y_t=g(h_t), the long-lag correlation is approximately C_y(k)=sum_j a_j z_j^k=sum_j a_j r_j^k exp(i k omega_j). Thus the predicted period is T_j=2 pi divided by omega_j, the decay time is tau_j=-1 divided by log(r_j), and the power spectrum has peaks near omega_j. Independently estimate the finite-time trajectory Lyapunov exponent Lambda_hat=(1/T) sum_t log(norm(J_t v_t)), where J_t is the hidden-state Jacobian and v_t is a repeatedly normalized tangent vector. Lambda_hat and log(r_j) measure different mechanisms.

## Key formulas

- $$(P_{theta} f)(h)=E_{x,xi}[f(F_{theta}(h,x,xi))],\qquad P_{theta} phi_j=z_j phi_j,\qquad z_j=r_j exp(i omega_j).$$
- $$C_y(k)approximately sum_j a_j z_j^k=sum_j a_j r_j^k exp(i k omega_j),\qquad T_j=2 pi divided by omega_j,\qquad tau_j=-1 divided by log(r_j).$$
- $$S_y(omega)=sum_{k=-infinity}^{infinity} C_y(k) exp(-i omega k),\qquad Lambda_hat=(1/T)sum_{t=0}^{T-1} log(norm(J_t v_t)),\qquad J_t=partial F_{theta}/partial h.$$
- $$L_res=sum_{j in J}[max(0,r_j-r_max)]^2+beta sum_{j in J} wrap(omega_j-omega_j^*)^2.$$

## Implementation notes

1. Integration point: use a tanh RNN, GRU, or diagonal state-space model with explicit Gaussian noise xi_t in the hidden update. During training, maintain a small ensemble of hidden trajectories under identical inputs but independent noise. Every M optimizer steps, collect one output coordinate, several hidden coordinates, and random projections of the hidden state. 2. Pseudocode: run B noisy trajectories; compute lag covariances C_hat(k)=B^{-1} sum_b (T-k)^{-1} sum_t (y_b,t-y_bar)(y_b,t+k-y_bar); fit m damped complex modes C_hat(k) approximately equal to sum_j a_j z_j^k using matrix-pencil, ESPRIT, or nonlinear least squares; calculate r_hat_j=abs(z_j) and omega_hat_j=arg(z_j); then add lambda times L_res to the task loss. Initially stop gradients through the fitted resonance estimates, or update the penalty only periodically. Estimate Lambda_hat with Jacobian-vector products and tangent-vector renormalization. 3. Computed versus estimated: the transfer-operator interpretation, resonance eigenvalue equation, and correlation expansion are the paper mechanism; finite-dimensional resonance values are estimated from sampled correlations rather than computed exactly. 4. First cheap experiment: train a noisy tanh RNN on a two-sine-wave prediction task and a copy-memory task, comparing ordinary weight decay, recurrent spectral-radius regularization, and resonance regularization. Sweep injected noise amplitude and recurrent gain. The falsifiable prediction is that a narrow output-spectrum peak appears when a fitted r_hat_j crosses approximately 0.95, with correlation decay time close to -1/log(r_hat_j), even when Lambda_hat remains negative. Penalizing r_hat_j above r_max=0.9 should remove that peak and reduce long-horizon error without necessarily changing Lambda_hat. The measured peak frequency should match omega_hat_j divided by 2 pi within about 10 percent.

## Verification

- Status: beats_baseline
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: helps
- Verdict: Built a readable stochastic rotational hidden-state MVP with correlation-based damped-mode fitting, PSD peak estimation, finite-time Lyapunov estimation, parameter sweeps, and resonance-radius control. The mechanism manifested quantitatively: fitted decay-time error was at most 2.9%, spectral peak error at most 4.1%, and fitted resonance parameters were invariant to noise while variance changed substantially. Controlling radius from 0.975 to 0.90 reduced peak power by 14.9x and long-lag correlation by 12.7x, although this is an exactly analyzable toy system rather than a trained nonlinear RNN.

### Mechanism check

- Verdict: Built a readable stochastic rotational hidden-state MVP with correlation-based damped-mode fitting, PSD peak estimation, finite-time Lyapunov estimation, parameter sweeps, and resonance-radius control. The mechanism manifested quantitatively: fitted decay-time error was at most 2.9%, spectral peak error at most 4.1%, and fitted resonance parameters were invariant to noise while variance changed substantially. Controlling radius from 0.975 to 0.90 reduced peak power by 14.9x and long-lag correlation by 12.7x, although this is an exactly analyzable toy system rather than a trained nonlinear RNN.
- Confidence: 8/10
- Limitations: The experiment uses a linear 2D noisy rotation, where the exact transfer resonance is known and Lyapunov estimation is straightforward; it does not train or compare nonlinear tanh/GRU RNNs on sine prediction or copy-memory tasks. The controller is implemented as direct recurrent-radius clipping rather than a differentiable periodically updated penalty, and no compute/FLOP or CUDA comparison was performed.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 8/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/447/bench_report.json)
- [bench_resonance.py](https://synthcore.org/code/447/bench_resonance.py)
- [report.md](https://synthcore.org/code/447/report.md)
- [report_bench_2026-08-31T122306.md](https://synthcore.org/code/447/report_bench_2026-08-31T122306.md)
- [resonance_experiment.py](https://synthcore.org/code/447/resonance_experiment.py)
- [results/report.json](https://synthcore.org/code/447/results/report.json)
- [results_stdout.json](https://synthcore.org/code/447/results_stdout.json)
- [Download all files as ZIP](https://synthcore.org/download/447)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.