△ Mechanism confirmed, baseline not beaten 2026

Explicit Multi-Timescale Memory Bank

Implementation & benchmark of arXiv:2608.04028 — Lindblad-Inspired Multi-Timescale Reservoir Computing with Separable Rotation and Dissipation

Usefulness7/10

Difficulty4/10

Novelty7/10

Architecture Accuracy Stability Sample efficiency Rnn Ssm Memory World model Dynamical systems Spectral theory Linear algebra Numerical analysis

Source paper: Lindblad-Inspired Multi-Timescale Reservoir Computing with Separable Rotation and Dissipation arXiv:2608.04028 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use a bank of damped rotational state channels with a deliberately spread decay spectrum, allowing one recurrent layer to represent short, medium, and long temporal dependencies without relying on a single learned spectral radius. Concatenate the channels and train a readout or downstream nonlinear head to select the appropriate memory timescale.

Formulas

$$B_j=e^{-\gamma_j}R(\omega_j),\qquad r_j=|\lambda_j|=e^{-\gamma_j},\qquad \tau_j\approx\frac{1}{\gamma_j}$$

$$\gamma_j=\exp\!\left(\log\gamma_{\min}+\frac{j-1}{K-1}\left(\log\gamma_{\max}-\log\gamma_{\min}\right)\right),\qquad j=1,\ldots,K$$

$$z_{t+1}^{(m)}=\tanh\!\left(W_m z_t^{(m)}+W_{\mathrm{in}}^{(m)}u_t+b_m\right),\qquad z_t=\operatorname{col}_{m=1}^{M}z_t^{(m)}$$

$$\|W_m\|_2=e^{-\min_j\gamma_{mj}}<1$$

Mathematical statement

For mode j, the scalar contraction factor is r_j = exp(-gamma_j), so an unforced linear mode retains information for a characteristic timescale approximately tau_j = 1/gamma_j when gamma_j is small. Its state rotates by omega_j radians per step while decaying by r_j. The paper's ensemble construction uses parallel members z_t^(m), each obeying z_{t+1}^(m) = tanh(W_m z_t^(m) + W_in^(m) u_t + b_m), followed by concatenation z_t = col_m z_t^(m). The transferable mechanism is to prescribe or learn a log-spaced set of gamma_j values, for example gamma_j in [gamma_min, gamma_max], so the memory bank covers exponentially separated horizons. The contraction bound for every member is ||W_m||_2 = exp(-min_j gamma_mj), while all modes remain bounded because exp(-gamma_j) <= 1. Frequencies omega_j control phase diversity independently of the memory horizon and can be sampled over a chosen interval such as [0, pi].

Implementation notes

Integrate this as a parallel recurrent feature extractor before a linear head, MLP head, or attention pooling layer. Split a fixed state budget d across M members or use one member containing K mode pairs. Initialize the decay rates log-uniformly over the target sequence horizons: choose tau_min and tau_max in timesteps, set gamma_min = 1/tau_max and gamma_max = 1/tau_min, then use the displayed geometric interpolation to obtain gamma_j. Sample omega_j uniformly in [0, pi] or use a deterministic frequency grid; construct every block B_j exactly from cosine and sine, and optionally apply a fixed orthogonal Q to mix coordinates. At each timestep, update all members in parallel, concatenate their states, and feed the result to the prediction head. Keep the bank fixed for the first experiment so improvements cannot be attributed to extra recurrent optimization; then unfreeze only gamma_j using gamma_j = softplus(rho_j) to test adaptive timescales. Compute the decay spectrum analytically and report the effective horizon of every mode; estimate empirical memory by linear-probe accuracy for reconstructing u_{t-k} at delays k. Start with a 128-state model on copy-memory, delayed XOR, NARMA-20, and Mackey-Glass, comparing against a same-width vanilla RNN and an orthogonal RNN. The expected signal is a flatter delay-reconstruction curve across short and long k, lower NARMA-20 error, and less degradation as required delay exceeds the median baseline memory horizon.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built an explicit parallel damped-rotation memory bank with log-spaced decay rates and a delayed-input linear probe. The three quantitative mechanism predictions were confirmed: decay-log slope matched -gamma to below 6e-17 absolute error, integer e-fold horizons matched 1/gamma exactly for four rates, and log-horizon spacing matched to 2.2e-16. In the toy probe, the bank improved long-delay MSE substantially at delays 32 and 64 (0.532 vs 0.897 and 0.867 vs 0.943), though the single-timescale baseline was better at several short delays, so the practical gain is horizon-dependent rather than universal.

Agent confidence: 9/10

Baseline: Single-timescale 32-state rotational reservoir delayed-probe MSE at delays [1,2,4,8,16,32,64,128,256]: [0.0616, 0.0618, 0.0633, 0.0657, 0.0775, 0.8975, 0.9426, 0.9552, 0.9501].
Idea: 16-mode, 32-state log-spaced bank delayed-probe MSE at delays [1,2,4,8,16,32,64,128,256]: [0.0195, 0.0983, 0.1927, 0.1928, 0.2029, 0.5319, 0.8671, 0.9331, 0.9452]. Math-check errors: slope <6e-17, e-fold 0, log-spacing 2.2e-16.

Limitations: Only a linear reservoir and linear readout on Gaussian delayed reconstruction were tested; no tanh recurrence, learned gamma values, nonlinear tasks such as NARMA/Mackey-Glass, orthogonal or trained vanilla RNN baseline, multiple seeds, FLOP/throughput measurement, or GPU implementation was evaluated.

How to run: python3 memory_bank_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.04028) Sep 2, 2026

Significantly WORSE than the tuned baseline (n/a, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0
Idea mean: 0.0058
Effect (Δ): 0.0043 (n/a; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Confirmed ✓
Practical effect: Hurts
Baseline tuning: swept over 0 configs

Limitations:

Only the built-in dynamics track was tested. Sequence forecasting, NARMA, Mackey-Glass, learned decay parameters, orthogonal-RNN alternatives, throughput, and larger state or epoch budgets were not tested. The signature measured trained hidden-state impulse decay rather than delay-reconstruction accuracy.

How to run: python3 stage2_memory_bank.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 10/10

Built a 64-state explicit log-spaced damped-rotation memory bank and compared it end-to-end with a same-width GRU on the matched dynamics track. The analytical decay and horizon-spacing checks passed, and trained hidden-state impulse decay was negative, but test MSE was significantly worse for the idea: 0.0057815 versus 0.0015080, paired delta +0.0042735, permutation p=0.0081. The mechanism transferred, but the benchmark result was a significant loss.

Baseline: Tuned GRU at lr=0.01: 8-seed dynamics test MSE mean 0.0015080313, std 0.0005411.
Idea: 64-state fixed log-spaced damped-rotation bank at lr=0.01: 8-seed dynamics test MSE mean 0.0057814980, std 0.0013363; paired delta +0.0042734668, p=0.0081.

Benchmark result