Idea description

Replace an unconstrained recurrent transition with a two-dimensional damped rotation whose parameters are induced by a learnable circular reorientation distribution. The first Fourier mode controls both memory persistence and phase rotation, giving the network an interpretable oscillatory memory while guaranteeing contraction when the effective decay rate is positive.

Formulas

$$\Pi_1=\int_{-\pi}^{\pi}q(\Delta\theta)e^{i\Delta\theta}\,d\Delta\theta,\qquad |\Pi_1|\leq1.$$

$$C(t)=e^{-\alpha(1-\operatorname{Re}\Pi_1)t}\cos\!\left(\alpha\operatorname{Im}(\Pi_1)t\right)=e^{-\gamma t}\cos(\Omega t).$$

$$h_t=e^{-\gamma\Delta t}R(\Omega\Delta t)h_{t-1}+Bx_t,\qquad R(\phi)=\begin{bmatrix}\cos\phi&-\sin\phi\\\sin\phi&\cos\phi\end{bmatrix}.$$

$$\left\|e^{-\gamma\Delta t}R(\Omega\Delta t)\right\|_2=e^{-\gamma\Delta t}<1\quad\Longleftrightarrow\quad\gamma>0.$$

Mathematical statement

Let q(\Delta heta) be a probability density over reorientation angles and define its first Fourier coefficient by $\Pi_1=\int_{-\pi}^{\pi}q(\Delta\theta)e^{i\Delta\theta}\,d\Delta\theta$, where $|\Pi_1|\leq1$. For tumble rate $\alpha$, the paper predicts the orientation correlation $C(t)=\exp[-\alpha(1-\operatorname{Re}\Pi_1)t]\cos[\alpha\operatorname{Im}(\Pi_1)t]$. Define effective damping $\gamma=\alpha(1-\operatorname{Re}\Pi_1)\geq0$ and angular frequency $\Omega=\alpha\operatorname{Im}(\Pi_1)$. A recurrent state $h_t\in\mathbb{R}^2$ uses the transition $h_t=\rho R(\phi)h_{t-1}+Bx_t$, with $\rho=e^{-\gamma\Delta t}\leq1$, $\phi=\Omega\Delta t$, and planar rotation matrix $R$. The homogeneous transition is contractive iff $\rho<1$, marginal at $\rho=1$, and unstable if $\rho>1$.

Implementation notes

Integrate this as a parallel recurrent channel inside an RNN, SSM, or transformer residual stream. Allocate several independent two-dimensional channels, each with its own circular distribution and therefore its own decay and frequency; concatenate their states with ordinary feed-forward features. Parameterize q with either a wrapped Gaussian, whose mean and concentration determine Pi_1, or a small softmax over discretized angles. At every update compute Pi_1, set gamma equal to alpha times one minus the real part of Pi_1, set Omega equal to alpha times the imaginary part, and apply the exact matrix exponential rather than Euler integration. Pseudocode is: Pi = sum(q_bins * exp(1j*angles)); gamma = alpha*(1-Pi.real); Omega = alpha*Pi.imag; rho = exp(-gamma*dt); h = rho*rot(Omega*dt)@h + B@x. The paper supplies the Fourier reduction and correlation law; training estimates q, alpha, B, and output parameters by backpropagation. First test on sequential-MNIST or the adding problem against tanh-RNN, GRU, and a diagonal SSM with matched parameter counts. Initialize one channel with Pi_1 approximately equal to 0.95 exp(0.2 i). The predicted hidden-state autocorrelation must have envelope slope minus gamma and oscillation frequency Omega, each within 20 percent of fitted traces. Sweeping the real part of Pi_1 toward one should increase the memory time as 1/gamma, while the recurrent Jacobian norm should remain at most one.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a Fourier-tumble recurrent channel using a discretized circular distribution and exact damped rotations. The mechanism manifested: sampled |Pi_1| stayed below 1, the Jacobian norm stayed below 1, the fitted correlation law matched predicted damping and frequency, and memory time increased as 1/gamma from 1.0 to 25.29 steps. The secondary fixed-parameter delayed-recall comparison did not show a baseline win, so this supports the mathematical mechanism rather than task-level superiority.

Agent confidence: 8/10

Baseline: Fixed-radius scalar retention delayed-recall MSE at delays 1,5,10,20,40,80: 0.0025, 0.0512, 0.1610, 0.4115, 0.7595, 0.9672.
Idea: Fourier channel delayed-recall MSE at delays 1,5,10,20,40,80: 0.0069, 0.3414, 1.3369, 1.4474, 0.9623, 1.0066. Maximum sampled |Pi_1|=0.9352, maximum Jacobian norm=0.9373, correlation-fit relative errors were zero, and memory 1/e time increased from 1.0 to 25.29.

Limitations: The correlation fit and memory check use the exact constructed transition, verifying implementation and numerical consistency rather than independently estimating a learned model. No backpropagation training, sequential-MNIST, GRU comparison, FLOP matching, noise robustness, or learned multi-channel task experiment was run.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.23519) Sep 3, 2026

No significant difference vs. the tuned baseline (+59.3%, p=0.2682, wins 3 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0005
Idea mean: 0.0009
Effect (Δ): 0.0003 (+59.3%; negative = idea better)
Wins: 3 / 8 paired seeds
p-value: 0.2682 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±115.9%
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

The official dynamics track was run successfully. The implementation uses a 64-dimensional Fourier channel system matching the rnn_small hidden width, but it does not include GRU or transformer comparisons because the required paired comparison is the standard rnn_small baseline versus the mechanism-modified recurrent system. The mechanism signature tests the trained transition behavior and contraction bound, not an independent task-level causal attribution.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 3, 2026 latest Failed ✗

Agent confidence: 10/10

The Fourier-tumble recurrent system was evaluated on the structurally matched official dynamics track using the registered rnn_small task, train_model, an equal learning-rate sweep, and 8 paired seeds. The idea's best mean test MSE was 0.000868575 versus the tuned baseline's 0.000545172, giving delta_mean +0.000323403 and permutation p=0.26825; therefore there is no significant win. The trained-model mechanism signature was confirmed: observed homogeneous transition ratio 0.799553, predicted norm 0.799553, and max |Pi_1| 0.131361, all within the contraction bound.

Baseline: Official dynamics baseline sweep: lr=0.001 mean 0.001864718, lr=0.003 mean 0.001090130, lr=0.01 mean 0.000618668 on sweep seeds; best lr=0.01 full 8-seed mean MSE 0.000545172.
Idea: Official dynamics idea sweep: lr=0.001 mean 0.002506814, lr=0.003 mean 0.001100955, lr=0.01 mean 0.000868575 on 8 seeds; best lr=0.01. Idea-minus-baseline paired delta +0.000323403, p=0.26825, 3/8 paired wins.
Mechanism signature: confirmed=true; trained-model max predicted transition norm 0.799553, max observed homogeneous ratio 0.799553, max |Pi_1| 0.131361.

Benchmark result