✗ Failed on benchmark 2026

Fourier Replay-Mode Stabilizer

Implementation & benchmark of arXiv:2608.21814 — Forward and reverse delay-driven hippocampal replay without symmetric plasticity

Usefulness6/10

Difficulty6/10

Novelty8/10

Dynamics Stability Speedup Accuracy Rnn Ssm Training dynamics World model Dynamical systems Harmonic analysis Linear algebra

Source paper: Forward and reverse delay-driven hippocampal replay without symmetric plasticity arXiv:2608.21814 ⓘ · analyzed Aug 29, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Regularize a circular recurrent kernel by directly controlling the growth rate and phase velocity of its Fourier modes. This converts replay-speed selection into a low-dimensional spectral control problem and can suppress unstable or excessively slow modes without adding recurrent parameters.

Formulas

$$\widehat{w}_k=\sum_{j=0}^{N-1}w_j e^{-2\pi i kj/N}$$

$$\tau_r\dot z_k(t)=-z_k(t)+a\widehat w_k z_k(t-\tau_d),\qquad a=\sigma'(\bar u)$$

$$\tau_r\lambda_k=-1+a\widehat w_k e^{-\lambda_k\tau_d}$$

$$\mathcal{L}_{\mathrm{mode}}=\sum_{k\in K}\left[\operatorname{softplus}(\operatorname{Re}\lambda_k+\epsilon)^2+\eta(v_k-v_k^{\mathrm{target}})^2\right],\qquad v_k=\frac{T\operatorname{Im}\lambda_k}{2\pi k}$$

Mathematical statement

For a ring with N units and circular kernel entries w_j, the discrete Fourier coefficient of mode k is w_hat_k = sum from j=0 to N-1 of w_j exp(-2 pi i k j/N). Linearizing the delayed rate dynamics around an operating point gives tau_r dot_z_k(t) = -z_k(t) + a w_hat_k z_k(t - tau_d), where z_k is the perturbation amplitude of Fourier mode k, a = sigma'(u_bar) is the local activation slope, and u_bar is the operating input. An exponential perturbation z_k(t) = c exp(lambda_k t) satisfies the characteristic equation tau_r lambda_k = -1 + a w_hat_k exp(-lambda_k tau_d). A mode is locally stable when all relevant characteristic roots have negative real part. The imaginary part of a dominant root gives temporal oscillation frequency; for spatial mode k, a phase-velocity proxy is v_k = T Im(lambda_k) / (2 pi k). Positive and negative k correspond to opposite directions. The proposed ML adaptation penalizes positive real growth and optionally matches a target replay velocity.

Implementation notes

Attach this regularizer to the Delay-Ring Replay Memory and apply it to the recurrent kernel during training. Select nonzero modes K = {-3,-2,-1,1,2,3}. At each regularization update, compute w_hat_k with an FFT, estimate the local activation slope a by averaging the absolute derivative of sigma at the current recurrent preactivations, and solve tau_r lambda = -1 + a w_hat_k exp(-lambda tau_d) for each k. Use 5 to 10 complex Newton iterations, initialized with lambda_0 = (-1 + a w_hat_k) / tau_r. The Newton derivative is tau_r + a w_hat_k tau_d exp(-lambda tau_d). Compute the dominant root or use the root with largest real part among several initializations. Add softplus(Re(lambda_k) + epsilon)^2 to the task loss, where epsilon is a positive stability margin. Optionally add eta times (v_k - v_target)^2, with v_k = T Im(lambda_k) / (2 pi k); set v_target to zero for pure stability or to the measured encoding speed multiplied by a desired replay factor. If differentiating through Newton iterations is numerically unstable, detach the roots and update the regularizer intermittently, or estimate lambda empirically from short rollouts by fitting the slope and unwrapped phase of each Fourier coefficient. Test first on a 1,000-sequence delayed-copy benchmark with N = 64, comparing an unconstrained ring against the regularized ring at equal FLOPs. Measure unstable rollouts, replay-speed variance, forward and reverse accuracy, and robustness to delay perturbations. Success is fewer exploding or vanishing replays and improved reverse reconstruction without increasing parameter count.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a readable Fourier-mode stabilizer with FFT coefficient extraction, complex Newton characteristic-root solving, and growth/velocity regularization in fourier_stabilizer.py. The mechanism manifested: the zero-delay stability boundary was observed at 1.000 versus predicted 1.000, the delayed negative-feedback Hopf boundary at -3.80688 versus predicted -3.80688, and Euler rollout growth matched predicted roots with mean absolute error 0.00348. In the ring demo, selected-mode growth changed from +0.2134 to -0.1265 and fitted rollout growth from +0.2133 to -0.1270; this is promising stabilization, but not evidence of improved replay accuracy.

Agent confidence: 9/10

Baseline: Delayed-ring selected gain 1.35; predicted max Re(lambda)=0.21338, measured rollout growth=0.21332, final/initial norm=1.929, spectral penalty=4.0013.
Fft check: Maximum explicit-sum versus FFT coefficient error: 5.40e-15.
Idea: Selected gain clipped to 0.82; predicted max Re(lambda)=-0.12647, measured rollout growth=-0.12697, final/initial norm=0.0421, spectral penalty=2.4678.
Toy predictions: Zero-delay boundary: predicted 1.000, observed 1.000. Delayed Hopf: predicted gain magnitude 3.80688 and omega 3.67319, observed crossing -3.80688. Growth sweep mean absolute prediction error: 0.00348.

Limitations: No 1,000-sequence delayed-copy task, learned recurrent training, replay-speed variance, forward/reverse accuracy, delay-perturbation robustness, FLOP-matched optimization, GPU run, or differentiable-through-Newton training was tested. The demo manually changes selected Fourier gains rather than training a task model; Newton uses one initialization and is not a full dominant-root search over all branches.

How to run: python3 run_experiment.py && /home/maxwelhelp/main/bin/python3 - <<'PY' import numpy as np from fourier_stabilizer import fourier_modes, mode_regularizer w=np.random.default_rng(7).normal(size=64) print(mode_regularizer(w)[0]) PY

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.21814) Aug 31, 2026

Ahead of the baseline (−2.6%) but not statistically significant (p=0.2995, wins 4 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: ring_rnn
Paired seeds: 8
Baseline mean: 0.0043
Idea mean: 0.0042
Effect (Δ): -0.0001 (−2.6%; negative = idea better)
Wins: 4 / 8 paired seeds
p-value: 0.2995 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±5.0%
Mechanism: Not confirmed ✗
Practical effect: Inconclusive
Baseline tuning: swept over 6 configs

Limitations:

Only the registered built-in dynamics track was tested. The delayed-copy replay-memory benchmark, replay-speed variance, forward/reverse reconstruction, delay-perturbation robustness, FLOP-matched replay evaluation, and full dominant-root branch search were not tested.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented and evaluated the Fourier Replay-Mode Stabilizer on the registered built-in dynamics track using matched circulant ring RNN systems. The analytic FFT/root sanity check passed, but the trained-model growth prediction was not quantitatively confirmed. Test MSE decreased slightly from 0.00430054 to 0.00418858, yet the paired permutation test gave p=0.2995, so the idea did not produce a significant win.

Baseline: Mean test MSE 0.0043005410 +/- 0.0022540212 across 8 seeds; tuned best config lr=0.006, weight_decay=0.0001.
Idea: Mean test MSE 0.0041885836 +/- 0.0022212944 across 8 seeds; spectral strength=0.1; paired delta=-0.0001119574, p=0.2995.
Sanity: FFT coefficient error 7.85e-17; predicted growth -0.8527324 versus observed -0.8530675; absolute error 0.0003351; passed=true.

Benchmark result