Memory ideas

External memories, state compression, retrieval-style read/write policies and calibrated stopping rules.

2 ideas found

Category: Memory ML: Optimizer Math: Statistics Clear all

Unverified 2026

Nonadaptive multiscale one-bit gradient sketch

Replace communicated floating-point gradients in synchronous federated or data-parallel training with one-bit threshold queries whose thresholds are sampled publicly before gradients are observed. Use several fixed geometric amplitude scales so the same protocol handles unknown gradient means and heavy-tailed client updates without an interactive localization round. Decode each coordinate from the scale whose neighboring estimates are statistically consistent, then apply the decoded aggregate…

Useful6/10

Difficulty6/10

Novelty6/10

Memory Memory Scalability Speedup

Paper: Interaction Is Not Necessary for Order-Optimal 1-Bit Mean Estimation arXiv:2608.02538

Unverified 2026

Decoder-Side Refinement for Heavy-Tailed Gradients

Replace ordinary sign-gradient or fixed-range gradient quantization in distributed training with a non-adaptive one-bit refinement block. Workers generate all localization and multiscale residue bits using public randomness before the server knows the current aggregate gradient; after decoding a coarse center, the server chooses which stored phase and scale interpretation to use. This should reduce clipping and variance for heavy-tailed gradients without requiring a second communication round.

Useful6/10

Difficulty6/10

Novelty7/10

Memory Memory Scalability Stability

Paper: Universal Refinement without Interaction: Order-Optimal 1-Bit Mean Estimation arXiv:2607.24358