✗ Failed on benchmark
2026
Learn a low-dimensional Koopman operator from successive states of an iterative neural system, such as debate agents, recurrent refinement blocks, or diffusion denoising trajectories. Use the magnitude of the subdominant eigenvalue to predict the remaining number of rounds required for disagreement to fall below a target tolerance, and stop computation when the predicted deadline is reached rather than using a fixed round budget.
Useful8/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace consecutive or randomly assigned transformed KV coefficients with groups whose variance-volume is approximately equal. Train one equal-size vector-quantizer codebook per group, so a fixed-width cache does not waste its low-rate budget by forcing high-variance and low-variance coordinates into badly mismatched groups. This is a drop-in quantization-layout change that can be applied to keys, values, or both.
Useful8/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Insert a linear Johnson–Lindenstrauss bottleneck around a set of jointly processed representations, choosing its width from the sharp finite-set dimension bound rather than from the model's nominal hidden size. The projection should preserve pairwise distances between tokens, patches, or retrieved items, allowing a downstream attention or MLP block to operate at lower width while retaining the geometry relevant to similarity computations.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Construct overlapping attention windows from a block design instead of using one dense sequence-by-sequence attention matrix. Every token appears in exactly $r$ windows and every token pair co-occurs in exactly $\lambda$ windows, giving uniform coverage and avoiding the uneven connectivity of arbitrary sparse masks.
Useful7/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Compress a transformer KV cache by selecting actual past tokens whose key or hidden-state columns form a stable basis for all cached tokens. Instead of retaining tokens with the largest attention scores or leverage scores independently, compute rank-revealing pivoting of the leading right-singular-vector matrix and retain its pivot columns, then evaluate attention using the representatives plus an optional low-cost residual correction.
Useful7/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Partition tokens into a rectangular grid and use Latin-square labels to define several sparse attention heads. Each head attends only to tokens sharing a row, column, or Latin label, while orthogonality guarantees that every pair of labels occurs at most once, reducing systematic blind spots and repeated collisions. The resulting masks are deterministic, reusable across examples, and can be generated without learned routing scores.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace quadratic self-attention over a sequence with a bank of K auxiliary exponentially decaying states whose rates are fitted directly from the empirical autocorrelation of the sequence features. Each mode captures a distinct time scale, so the module can represent short- and long-range dependencies with O(TK) computation and O(K) recurrent memory rather than storing all previous tokens. Constrain decay rates to be positive and use the paper's extended Markovian block structure to obtain a…
Useful7/10
Difficulty5/10
Novelty4/10
✗ Failed on benchmark
2026
Use a smoothed Burg entropy as the mirror map in a proximal-gradient optimizer for positive or simplex-valued neural parameters. The optimizer performs a Bregman-proximal step instead of an additive Euclidean update, while the smoothing parameter avoids the singularity of ordinary Burg entropy at zero.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Construct a sparse token-interaction graph from attention affinities and recursively split it by removing the closed neighborhoods of a small set of separator tokens. Separator tokens retain global communication, while the resulting anti-adjacent components perform local attention independently, reducing quadratic attention and KV-cache costs. The induced Erdos-Pósa theorem supplies a structural diagnostic: graphs with few anti-adjacent long-cycle or theta packings should admit small…
Useful6/10
Difficulty7/10
Novelty8/10