△ Mechanism confirmed, baseline not beaten
2026
Allocate different entropy budgets to different KV-cache blocks instead of assigning every token and head the same nominal bitwidth. Use the ECASQ Lagrangian so high-variance or attention-sensitive blocks receive more codepoints, while predictable blocks collapse to fewer symbols and become highly compressible. Preserve unbiasedness per scalar or block so reconstructed keys and values have zero mean quantization error conditional on the original tensor.
Useful8/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace the usual random or elementwise-positive linear-attention feature map with a rank-one positive-semidefinite feature map derived from query and key vectors. For normalized inputs, the resulting kernel is the squared inner product, which is nonnegative and gives a geometrically structured interference pattern that is better suited to associative recall than an arbitrary low-rank feature map.
Useful8/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Compress each layer's KV tensor with a partial Tucker approximation over token and feature axes, then encode the truncation residual with a rotated uniform quantizer. Select token rank, feature rank, and residual bit-width jointly under a global byte budget, allowing values with flat spectra to receive residual bits while keys may receive more low-rank capacity.
Useful8/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Compress a causal sequence by retaining history positions at equal increments of cumulative representation variation instead of at uniform time intervals. Use the resulting N representatives in a decoder that reconstructs piecewise-constant keys and values; the paper's minimax result predicts a worst-case reconstruction error of total variation divided by 2N, independent of where rapid changes occur.
Useful8/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace token-by-token KV storage after an SSM or recurrent encoder with an online allocate-on-novelty cache. A new slot is created only when the incoming key is sufficiently dissimilar from every stored key; otherwise the incoming value is merged into its nearest slot, so repeated or redundant content does not grow the cache.
Useful8/10
Difficulty4/10
Novelty6/10
✗ Mechanism failed
2026
Replace the usual JL dimension choice with a task-specific dimension budget based on the paper's asymptotic ranking law. For retrieval, nearest-neighbor search, or compressed attention keys, choose m from a target Kendall correlation and automatically expand the projection when measured ranking quality falls below the target.
Useful7/10
Difficulty3/10
Novelty7/10
✗ Failed on benchmark
2026
Replace unrestricted continuous batching with a fairness-constrained scheduler that only inserts a waiting request when its predicted decode footprint remains within a normalized distance of the longest request already in the batch. The fairness budget should be tuned away from the mathematically worst midpoint and validated against real arrival distributions, with the goal of reducing wasted max-driven decode work without falling back to inefficient one-request-at-a-time serving.
Useful7/10
Difficulty4/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace dense attention between tokens on opposite sides of a one-dimensional boundary or segment split with a dyadic low-rank approximation of a Cauchy/Hankel distance kernel. Each distance-scale block uses O(log(1/\varepsilon)) features, and the number of active scales grows only logarithmically with context length after discarding a narrow boundary layer. This is especially suitable for a relative-position attention branch or state-space-like long-range branch, rather than arbitrary…
Useful7/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Construct overlapping attention windows from a block design instead of using one dense sequence-by-sequence attention matrix. Every token appears in exactly $r$ windows and every token pair co-occurs in exactly $\lambda$ windows, giving uniform coverage and avoiding the uneven connectivity of arbitrary sparse masks.
Useful7/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Construct a sparse attention layer by sampling backward token histories as a continuous-time branching process rather than allowing every query to attend to every key. Each active ancestor either dies or branches into a bounded number of candidate ancestors, with branching probability controlled by a small parameter. The branch-out penalty predicts exponentially small probability of long, highly branching histories, providing a direct knob for receptive-field size and attention FLOPs.
Useful7/10
Difficulty5/10
Novelty7/10
Unverified
2026
Use the number of tokens participating in an attention operation to adapt the projected query/key dimension according to the sharp Johnson–Lindenstrauss bound. Short attention windows receive a dimension limited by n-1, while longer windows use a logarithmic dimension, avoiding a fixed large head dimension when the current attention set contains too few points to require it.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Approximate a long hidden-state or key/value trajectory by a small sum of exponential modes, storing only the mode nodes and vector weights instead of every timestep. Recover the modes from a Hankel matrix through an annihilating polynomial, then reconstruct the trajectory or a compact recurrent state during inference.
Useful6/10
Difficulty6/10
Novelty5/10
Unverified
2026
Use the paper's effective-rank and eigengap-dependent covariance estimation rates to construct a confidence-aware low-rank bottleneck for transformer activations or key/value tensors. The bottleneck is enabled only when the top-p empirical eigenspace can be estimated more accurately than the desired compression error; otherwise the layer remains full-rank.
Useful6/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Construct a sparse token-interaction graph from attention affinities and recursively split it by removing the closed neighborhoods of a small set of separator tokens. Separator tokens retain global communication, while the resulting anti-adjacent components perform local attention independently, reducing quadratic attention and KV-cache costs. The induced Erdos-Pósa theorem supplies a structural diagnostic: graphs with few anti-adjacent long-cycle or theta packings should admit small…
Useful6/10
Difficulty7/10
Novelty8/10
△ Mechanism confirmed, baseline not beaten
2026
Replace dense token-to-token attention by a learned binary relation generated from a small number of hierarchical predicates, while rejecting masks that contain a fixed K_{t,t} biclique. The paper's incidence bound predicts near-linear active edges for these structured relations, giving sparse attention with a measurable worst-case complexity target rather than relying only on average sparsity.
Useful6/10
Difficulty6/10
Novelty6/10