Memory ideas

External memories, state compression, retrieval-style read/write policies and calibrated stopping rules.

2 ideas found

Category: Memory Solves: Stability Math: Statistics Clear all

✗ Failed on benchmark 2026

Response-Sufficient Neural Memory

Replace correlation-based memory pruning in an RNN or state-space model by measuring how hidden-state history changes the response to individual past input events. Train a compressed memory coordinate only if it preserves the event-consequence kernel for the target observable, such as future loss, prediction, or control return. A memory representation is accepted when the conditional variance of this kernel within compressed-state groups is small, even if dwell-time or autocorrelation…

Useful8/10

Difficulty6/10

Novelty8/10

Memory Memory Accuracy Stability

Paper: The Memory Hidden in Response Fluctuations: Trajectory-Level Fluctuation-Response Theory and Inequalities for Non-Markovian Jump Dynamics arXiv:2608.20328

Unverified 2026

Decoder-Side Refinement for Heavy-Tailed Gradients

Replace ordinary sign-gradient or fixed-range gradient quantization in distributed training with a non-adaptive one-bit refinement block. Workers generate all localization and multiscale residue bits using public randomness before the server knows the current aggregate gradient; after decoding a coarse center, the server chooses which stored phase and scale interpretation to use. This should reduce clipping and variance for heavy-tailed gradients without requiring a second communication round.

Useful6/10

Difficulty6/10

Novelty7/10

Memory Memory Scalability Stability

Paper: Universal Refinement without Interaction: Order-Optimal 1-Bit Mean Estimation arXiv:2607.24358