Anomalous diffusion memory factorization: Characteristic timescales and application to inverse problem
arXiv:2608.21674
2026
Dynamics
2 ideas extracted · analyzed Sep 1, 2026
What the math gives to ML
The paper identifies Mittag-Leffler relaxation as a constructive alternative to exponential forgetting: for fractional order \(0<\alpha<1\), its tail decays as a power law \(t^{-\alpha}\), retaining information over many timescales. This is transferable to neural systems because standard exponential moving averages, recurrent states, and linear state-space models have geometrically decaying memory and therefore forget old signals too quickly. The most practical adaptation is to approximate a Mittag-Leffler kernel with a small bank of exponential states, preserving long memory while retaining \(O(KT)\) computation and constant-size recurrent state. This can be tested both as a multi-timescale optimizer accumulator and as a long-context recurrent or SSM layer.
Ideas from this paper
✗ Mechanism failed
2026
Construct an efficient recurrent or state-space layer whose impulse response follows Mittag-Leffler relaxation instead of a single exponential. A bank of stable diagonal state channels approximates the long power-law tail, allowing the layer to retain information over widely separated timescales with only \(K\) states per feature.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Replace AdamW's single exponentially decaying second-moment accumulator with a small bank of accumulators whose combined impulse response approximates fractional relaxation. The resulting preconditioner remembers rare or old gradient directions with a power-law rather than geometric decay, which may improve optimization on nonstationary, sparse-gradient, or long-horizon problems.
Useful7/10
Difficulty4/10
Novelty6/10