# Fourier Replay-Mode Stabilizer

- ID: 42
- Canonical URL: https://synthcore.org/idea/42/fourier-replay-mode-stabilizer
- API JSON: https://synthcore.org/api/idea/42.json
- API Markdown: https://synthcore.org/api/idea/42.md
- Verification status: failed_benchmark
- Source: [arXiv:2608.21814](https://arxiv.org/abs/2608.21814)
- Category: dynamics
- Solves: stability, speedup, accuracy
- ML areas: rnn, ssm, training-dynamics, world-model
- Math tags: dynamical-systems, harmonic-analysis, linear-algebra
- Ratings: usefulness 6/10; difficulty 6/10; novelty 8/10

## Idea description

Regularize a circular recurrent kernel by directly controlling the growth rate and phase velocity of its Fourier modes. This converts replay-speed selection into a low-dimensional spectral control problem and can suppress unstable or excessively slow modes without adding recurrent parameters.

## Mathematical statement

For a ring with N units and circular kernel entries w_j, the discrete Fourier coefficient of mode k is w_hat_k = sum from j=0 to N-1 of w_j exp(-2 pi i k j/N). Linearizing the delayed rate dynamics around an operating point gives tau_r dot_z_k(t) = -z_k(t) + a w_hat_k z_k(t - tau_d), where z_k is the perturbation amplitude of Fourier mode k, a = sigma'(u_bar) is the local activation slope, and u_bar is the operating input. An exponential perturbation z_k(t) = c exp(lambda_k t) satisfies the characteristic equation tau_r lambda_k = -1 + a w_hat_k exp(-lambda_k tau_d). A mode is locally stable when all relevant characteristic roots have negative real part. The imaginary part of a dominant root gives temporal oscillation frequency; for spatial mode k, a phase-velocity proxy is v_k = T Im(lambda_k) / (2 pi k). Positive and negative k correspond to opposite directions. The proposed ML adaptation penalizes positive real growth and optionally matches a target replay velocity.

## Key formulas

- $$\widehat{w}_k=\sum_{j=0}^{N-1}w_j e^{-2\pi i kj/N}$$
- $$\tau_r\dot z_k(t)=-z_k(t)+a\widehat w_k z_k(t-\tau_d),\qquad a=\sigma'(\bar u)$$
- $$\tau_r\lambda_k=-1+a\widehat w_k e^{-\lambda_k\tau_d}$$
- $$\mathcal{L}_{\mathrm{mode}}=\sum_{k\in K}\left[\operatorname{softplus}(\operatorname{Re}\lambda_k+\epsilon)^2+\eta(v_k-v_k^{\mathrm{target}})^2\right],\qquad v_k=\frac{T\operatorname{Im}\lambda_k}{2\pi k}$$

## Implementation notes

Attach this regularizer to the Delay-Ring Replay Memory and apply it to the recurrent kernel during training. Select nonzero modes K = {-3,-2,-1,1,2,3}. At each regularization update, compute w_hat_k with an FFT, estimate the local activation slope a by averaging the absolute derivative of sigma at the current recurrent preactivations, and solve tau_r lambda = -1 + a w_hat_k exp(-lambda tau_d) for each k. Use 5 to 10 complex Newton iterations, initialized with lambda_0 = (-1 + a w_hat_k) / tau_r. The Newton derivative is tau_r + a w_hat_k tau_d exp(-lambda tau_d). Compute the dominant root or use the root with largest real part among several initializations. Add softplus(Re(lambda_k) + epsilon)^2 to the task loss, where epsilon is a positive stability margin. Optionally add eta times (v_k - v_target)^2, with v_k = T Im(lambda_k) / (2 pi k); set v_target to zero for pure stability or to the measured encoding speed multiplied by a desired replay factor. If differentiating through Newton iterations is numerically unstable, detach the roots and update the regularizer intermittently, or estimate lambda empirically from short rollouts by fitting the slope and unwrapped phase of each Fourier coefficient. Test first on a 1,000-sequence delayed-copy benchmark with N = 64, comparing an unconstrained ring against the regularized ring at equal FLOPs. Measure unstable rollouts, replay-speed variance, forward and reverse accuracy, and robustness to delay perturbations. Success is fewer exploding or vanishing replays and improved reverse reconstruction without increasing parameter count.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: inconclusive
- Verdict: Built a readable Fourier-mode stabilizer with FFT coefficient extraction, complex Newton characteristic-root solving, and growth/velocity regularization in fourier_stabilizer.py. The mechanism manifested: the zero-delay stability boundary was observed at 1.000 versus predicted 1.000, the delayed negative-feedback Hopf boundary at -3.80688 versus predicted -3.80688, and Euler rollout growth matched predicted roots with mean absolute error 0.00348. In the ring demo, selected-mode growth changed from +0.2134 to -0.1265 and fitted rollout growth from +0.2133 to -0.1270; this is promising stabilization, but not evidence of improved replay accuracy.

### Mechanism check

- Verdict: Built a readable Fourier-mode stabilizer with FFT coefficient extraction, complex Newton characteristic-root solving, and growth/velocity regularization in fourier_stabilizer.py. The mechanism manifested: the zero-delay stability boundary was observed at 1.000 versus predicted 1.000, the delayed negative-feedback Hopf boundary at -3.80688 versus predicted -3.80688, and Euler rollout growth matched predicted roots with mean absolute error 0.00348. In the ring demo, selected-mode growth changed from +0.2134 to -0.1265 and fitted rollout growth from +0.2133 to -0.1270; this is promising stabilization, but not evidence of improved replay accuracy.
- Confidence: 9/10
- Limitations: No 1,000-sequence delayed-copy task, learned recurrent training, replay-speed variance, forward/reverse accuracy, delay-perturbation robustness, FLOP-matched optimization, GPU run, or differentiable-through-Newton training was tested. The demo manually changes selected Fourier gains rather than training a task model; Newton uses one initialization and is not a full dominant-root search over all branches.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.2995
- Paired wins: 4/8
- Benchmark verdict: no significant win

## Artifacts

- [bench_report.json](https://synthcore.org/code/692/bench_report.json)
- [final_check.json](https://synthcore.org/code/692/final_check.json)
- [fourier_stabilizer.py](https://synthcore.org/code/692/fourier_stabilizer.py)
- [report.md](https://synthcore.org/code/692/report.md)
- [report_bench_2026-08-31T221413.md](https://synthcore.org/code/692/report_bench_2026-08-31T221413.md)
- [results.json](https://synthcore.org/code/692/results.json)
- [run_experiment.py](https://synthcore.org/code/692/run_experiment.py)
- [stage2_bench.py](https://synthcore.org/code/692/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/692)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.