# Fourier-Tumble Oscillatory Memory

- ID: 2703
- Canonical URL: https://synthcore.org/idea/2703/fourier-tumble-oscillatory-memory
- API JSON: https://synthcore.org/api/idea/2703.json
- API Markdown: https://synthcore.org/api/idea/2703.md
- Verification status: unverified
- Source: [arXiv:2608.23519](https://arxiv.org/abs/2608.23519)
- Category: dynamics
- Solves: stability, accuracy, memory
- ML areas: rnn, ssm, training-dynamics, initialization
- Math tags: dynamical-systems, statistical-mechanics, spectral-theory, probability
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Replace an unconstrained recurrent transition with a two-dimensional damped rotation whose parameters are induced by a learnable circular reorientation distribution. The first Fourier mode controls both memory persistence and phase rotation, giving the network an interpretable oscillatory memory while guaranteeing contraction when the effective decay rate is positive.

## Mathematical statement

Let q(\Delta    heta) be a probability density over reorientation angles and define its first Fourier coefficient by \(\Pi_1=\int_{-\pi}^{\pi}q(\Delta\theta)e^{i\Delta\theta}\,d\Delta\theta\), where \(|\Pi_1|\leq1\). For tumble rate \(\alpha\), the paper predicts the orientation correlation \(C(t)=\exp[-\alpha(1-\operatorname{Re}\Pi_1)t]\cos[\alpha\operatorname{Im}(\Pi_1)t]\). Define effective damping \(\gamma=\alpha(1-\operatorname{Re}\Pi_1)\geq0\) and angular frequency \(\Omega=\alpha\operatorname{Im}(\Pi_1)\). A recurrent state \(h_t\in\mathbb{R}^2\) uses the transition \(h_t=\rho R(\phi)h_{t-1}+Bx_t\), with \(\rho=e^{-\gamma\Delta t}\leq1\), \(\phi=\Omega\Delta t\), and planar rotation matrix \(R\). The homogeneous transition is contractive iff \(\rho<1\), marginal at \(\rho=1\), and unstable if \(\rho>1\).

## Key formulas

- $$\Pi_1=\int_{-\pi}^{\pi}q(\Delta\theta)e^{i\Delta\theta}\,d\Delta\theta,\qquad |\Pi_1|\leq1.$$
- $$C(t)=e^{-\alpha(1-\operatorname{Re}\Pi_1)t}\cos\!\left(\alpha\operatorname{Im}(\Pi_1)t\right)=e^{-\gamma t}\cos(\Omega t).$$
- $$h_t=e^{-\gamma\Delta t}R(\Omega\Delta t)h_{t-1}+Bx_t,\qquad R(\phi)=\begin{bmatrix}\cos\phi&-\sin\phi\\\sin\phi&\cos\phi\end{bmatrix}.$$
- $$\left\|e^{-\gamma\Delta t}R(\Omega\Delta t)\right\|_2=e^{-\gamma\Delta t}<1\quad\Longleftrightarrow\quad\gamma>0.$$

## Implementation notes

Integrate this as a parallel recurrent channel inside an RNN, SSM, or transformer residual stream. Allocate several independent two-dimensional channels, each with its own circular distribution and therefore its own decay and frequency; concatenate their states with ordinary feed-forward features. Parameterize q with either a wrapped Gaussian, whose mean and concentration determine Pi_1, or a small softmax over discretized angles. At every update compute Pi_1, set gamma equal to alpha times one minus the real part of Pi_1, set Omega equal to alpha times the imaginary part, and apply the exact matrix exponential rather than Euler integration. Pseudocode is: `Pi = sum(q_bins * exp(1j*angles)); gamma = alpha*(1-Pi.real); Omega = alpha*Pi.imag; rho = exp(-gamma*dt); h = rho*rot(Omega*dt)@h + B@x`. The paper supplies the Fourier reduction and correlation law; training estimates q, alpha, B, and output parameters by backpropagation. First test on sequential-MNIST or the adding problem against tanh-RNN, GRU, and a diagonal SSM with matched parameter counts. Initialize one channel with Pi_1 approximately equal to 0.95 exp(0.2 i). The predicted hidden-state autocorrelation must have envelope slope minus gamma and oscillation frequency Omega, each within 20 percent of fitted traces. Sweeping the real part of Pi_1 toward one should increase the memory time as 1/gamma, while the recurrent Jacobian norm should remain at most one.

## Verification

- Status: unverified
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a Fourier-tumble recurrent channel using a discretized circular distribution and exact damped rotations. The mechanism manifested: sampled |Pi_1| stayed below 1, the Jacobian norm stayed below 1, the fitted correlation law matched predicted damping and frequency, and memory time increased as 1/gamma from 1.0 to 25.29 steps. The secondary fixed-parameter delayed-recall comparison did not show a baseline win, so this supports the mathematical mechanism rather than task-level superiority.

### Mechanism check

- Verdict: Built a Fourier-tumble recurrent channel using a discretized circular distribution and exact damped rotations. The mechanism manifested: sampled |Pi_1| stayed below 1, the Jacobian norm stayed below 1, the fitted correlation law matched predicted damping and frequency, and memory time increased as 1/gamma from 1.0 to 25.29 steps. The secondary fixed-parameter delayed-recall comparison did not show a baseline win, so this supports the mathematical mechanism rather than task-level superiority.
- Confidence: 8/10
- Limitations: The correlation fit and memory check use the exact constructed transition, verifying implementation and numerical consistency rather than independently estimating a learned model. No backpropagation training, sequential-MNIST, GRU comparison, FLOP matching, noise robustness, or learned multi-channel task experiment was run.

## Artifacts

- [README.md](https://synthcore.org/code/1042/README.md)
- [experiment.py](https://synthcore.org/code/1042/experiment.py)
- [report.md](https://synthcore.org/code/1042/report.md)
- [results.json](https://synthcore.org/code/1042/results.json)
- [Download all files as ZIP](https://synthcore.org/download/1042)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.