A Logarithmic Fluctuation Hierarchy for Sequential Interacting Diffusions
arXiv:2607.22470
2026
Architecture
1 ideas extracted · analyzed Aug 30, 2026
What the math gives to ML
The paper shows that systems with the same deterministic mean-field limit can retain different order-sensitive information at the Gaussian fluctuation scale. Its transferable construction is a finite hierarchy of log-rank weighted prefix summaries, where successive channels preserve increasingly high moments of sequential position. These summaries can be computed online in linear time and used as a low-cost complement to causal attention or as a memory-efficient state in a sequence model. The most direct test is to add them to a small causal Transformer and evaluate long-range prediction at matched compute and memory.
Ideas from this paper
Unverified
2026
Augment a causal sequence model with a small hierarchy of prefix summaries weighted by powers of the logarithmic rank of each preceding token. The summaries retain order-sensitive deviations from a baseline representation while costing O(KNd) for sequence length N, hierarchy width K, and hidden dimension d, instead of O(N^2d) dense attention.
Useful6/10
Difficulty4/10
Novelty6/10