Memory ideas

External memories, state compression, retrieval-style read/write policies and calibrated stopping rules.

2 ideas found

Status: Failed on benchmark Category: Memory ML: Attention Clear all

✗ Failed on benchmark 2026

Equal-Volume KV Vector Quantization

Replace consecutive or randomly assigned transformed KV coefficients with groups whose variance-volume is approximately equal. Train one equal-size vector-quantizer codebook per group, so a fixed-width cache does not waste its low-rate budget by forcing high-variance and low-variance coordinates into badly mismatched groups. This is a drop-in quantization-layout change that can be applied to keys, values, or both.

Useful8/10

Difficulty5/10

Novelty7/10

Memory Memory Speedup Accuracy

Paper: Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms arXiv:2608.04074

✗ Failed on benchmark 2026

Rank-Revealing Representative Tokens

Compress a transformer KV cache by selecting actual past tokens whose key or hidden-state columns form a stable basis for all cached tokens. Instead of retaining tokens with the largest attention scores or leverage scores independently, compute rank-revealing pivoting of the leading right-singular-vector matrix and retain its pivot columns, then evaluate attention using the representatives plus an optional low-cost residual correction.

Useful7/10

Difficulty5/10

Novelty6/10

Memory Memory Speedup Accuracy

Paper: Computing Strong Rank-Revealing Factorizations for Matrices with Orthonormal Rows arXiv:2607.13532