# Delay-Ring Replay Memory

- ID: 41
- Canonical URL: https://synthcore.org/idea/41/delay-ring-replay-memory
- API JSON: https://synthcore.org/api/idea/41.json
- API Markdown: https://synthcore.org/api/idea/41.md
- Verification status: mechanism_failed
- Source: [arXiv:2608.21814](https://arxiv.org/abs/2608.21814)
- Category: architecture
- Solves: memory, sample-efficiency, accuracy
- ML areas: rnn, ssm, world-model
- Math tags: dynamical-systems, harmonic-analysis, linear-algebra
- Ratings: usefulness 7/10; difficulty 5/10; novelty 8/10

## Idea description

Replace a directed sequence-memory chain with a circular recurrent state propagated by a learned delayed convolution. The same learned kernel can support forward and reverse replay because replay direction is a dynamical mode of the ring, rather than requiring plasticity to explicitly learn both forward and backward synapses.

## Mathematical statement

The state is a firing-rate function r(x,t), where x is a circular position in [0,T), t is time, and T is the ring period. The neural-field dynamics are tau_r partial_t r(x,t) = -r(x,t) + sigma(u(x,t) + [w * r_pre](x,t)), where tau_r > 0 is the rate time constant, u is external input, sigma is the activation function, and r_pre(x,t) = r(x,t - tau_d) uses delay tau_d. The delayed circular convolution is [w * r_pre](x,t) = integral from 0 to T of w(Delta x,t) r(x - Delta x,t - tau_d) d(Delta x), where w(Delta x,t) is the recurrent weight at phase offset Delta x. The asymmetric plasticity rule is tau_w partial_t w(x,y,t) = r(y,t - tau_d) partial_t r(x,t) - gamma w(x,y,t), where tau_w > 0 is the plasticity time constant and gamma > 0 is weight decay. The key transferable property is translation-equivariance on a ring: Fourier modes exp(i 2 pi k x/T) are eigenmodes of circular convolution, with positive and negative k representing opposite propagation directions.

## Key formulas

- $$\tau_r\partial_t r(x,t)=-r(x,t)+\sigma\left(u(x,t)+[w*r_{\mathrm{pre}}](x,t)\right)$$
- $$[w*r_{\mathrm{pre}}](x,t)=\int_0^T w(\Delta x,t)r(x-\Delta x,t-\tau_d)\,d\Delta x,\qquad r_{\mathrm{pre}}(x,t)=r(x,t-\tau_d)$$
- $$\tau_w\partial_t w(x,y,t)=r(y,t-\tau_d)\partial_t r(x,t)-\gamma w(x,y,t)$$
- $$\widehat{w}_k=\sum_{j=0}^{N-1}w_j e^{-2\pi i kj/N},\qquad \widehat{c}_{k,t+1}=\widehat{w}_k\widehat{c}_{k,t-d}$$

## Implementation notes

Integrate this as a recurrent memory block between an encoder and decoder in a small sequence model or world model. Use N ring units r_t, a circular kernel w with N offsets, and a delay buffer of length d = ceil(tau_d / Delta t). During each step, read the delayed state r_{t-d}, compute q_t = circular_convolution(w, r_{t-d}), and update r_{t+1} = r_t + (Delta t / tau_r) * (-r_t + sigma(u_t + q_t)). During an online learning phase, compute dot_r_t = (r_{t+1} - r_t) / Delta t and update each offset by the discrete version of the paper rule: w_j <- (1 - Delta t gamma / tau_w) w_j + (Delta t / tau_w) Delta_w_j, where Delta_w_j = (1/N) sum_i r_{t-d,i} dot_r_{t,(i+j) mod N}. Normalize or clip w after each update to prevent runaway excitation. Implement the circular convolution with FFT as q = IFFT(FFT(w) elementwise-multiplied by FFT(r_{t-d})), reducing cost from O(N^2) to O(N log N). The paper supplies the delayed convolution and derivative-Hebbian update; tau_r, tau_d, tau_w, gamma, activation, and initialization are hyperparameters estimated by validation. First test on a synthetic delayed-copy task containing 1,000 noisy sequences. Present each sequence once, then provide blank input and measure replay in both directions. Compare with a GRU, a directed Hebbian chain, and a symmetric recurrent ring at equal parameter count. Report forward and reverse reconstruction accuracy, retention after increasing blank intervals, replay speed, and robustness to noise. Success means higher bidirectional replay accuracy after one exposure and lower recurrent cost at the same ring width.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a compact delayed circular-ring replay MVP with FFT convolution, kernel fitting, replay evaluation, retention testing, and diagnostic controls. The convolution and Fourier eigenmode identities held to about 1e-14, and FFT convolution was about 256x faster than direct convolution at N=256. However, one learned clockwise kernel did not replay the sequence in reverse: reverse correlation was 0.035 with 11.1% accuracy, while an explicitly reflected kernel reached 0.996 correlation and 100% accuracy but is a distinct operator. Retention also collapsed after blank intervals, so the claimed bidirectional transfer effect was not observed.

### Mechanism check

- Verdict: Built a compact delayed circular-ring replay MVP with FFT convolution, kernel fitting, replay evaluation, retention testing, and diagnostic controls. The convolution and Fourier eigenmode identities held to about 1e-14, and FFT convolution was about 256x faster than direct convolution at N=256. However, one learned clockwise kernel did not replay the sequence in reverse: reverse correlation was 0.035 with 11.1% accuracy, while an explicitly reflected kernel reached 0.996 correlation and 100% accuracy but is a distinct operator. Retention also collapsed after blank intervals, so the claimed bidirectional transfer effect was not observed.
- Confidence: 9/10
- Limitations: The experiment used a small controlled moving-bump sequence rather than the proposed 1,000-sequence delayed-copy benchmark. It did not compare a GRU, implement the online derivative-Hebbian plasticity rule or explicit delay buffer, measure equal-FLOP training, or test learned dynamical mode switching.

## Artifacts

- [delay_ring_experiment.py](https://synthcore.org/code/57/delay_ring_experiment.py)
- [diagnostic.py](https://synthcore.org/code/57/diagnostic.py)
- [report.md](https://synthcore.org/code/57/report.md)
- [results.json](https://synthcore.org/code/57/results.json)
- [Download all files as ZIP](https://synthcore.org/download/57)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.