Memory ideas

External memories, state compression, retrieval-style read/write policies and calibrated stopping rules.

2 ideas found

Category: Memory ML: Attention Math: Statistics Clear all

✗ Failed on benchmark 2026

Equal-Volume KV Vector Quantization

Replace consecutive or randomly assigned transformed KV coefficients with groups whose variance-volume is approximately equal. Train one equal-size vector-quantizer codebook per group, so a fixed-width cache does not waste its low-rate budget by forcing high-variance and low-variance coordinates into badly mismatched groups. This is a drop-in quantization-layout change that can be applied to keys, values, or both.

Useful8/10

Difficulty5/10

Novelty7/10

Memory Memory Speedup Accuracy

Paper: Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms arXiv:2608.04074

✗ Mechanism failed 2026

Ranking-Aware Projection Dimension Rule

Replace the usual JL dimension choice with a task-specific dimension budget based on the paper's asymptotic ranking law. For retrieval, nearest-neighbor search, or compressed attention keys, choose m from a target Kendall correlation and automatically expand the projection when measured ranking quality falls below the target.

Useful7/10

Difficulty3/10

Novelty7/10

Memory Memory Accuracy Scalability

Paper: Exact Limits of Random Projections for Preserving Geometry: Distance Recovery, Nearest-Neighbor Rankings, and Covariance Shape in Gaussian Models arXiv:2609.02155