△ Mechanism confirmed, baseline not beaten
2026
Allocate different entropy budgets to different KV-cache blocks instead of assigning every token and head the same nominal bitwidth. Use the ECASQ Lagrangian so high-variance or attention-sensitive blocks receive more codepoints, while predictable blocks collapse to fewer symbols and become highly compressible. Preserve unbiasedness per scalar or block so reconstructed keys and values have zero mean quantization error conditional on the original tensor.
Useful8/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace token-by-token KV storage after an SSM or recurrent encoder with an online allocate-on-novelty cache. A new slot is created only when the incoming key is sufficiently dissimilar from every stored key; otherwise the incoming value is merged into its nearest slot, so repeated or redundant content does not grow the cache.
Useful8/10
Difficulty4/10
Novelty6/10
✗ Mechanism failed
2026
Replace the usual JL dimension choice with a task-specific dimension budget based on the paper's asymptotic ranking law. For retrieval, nearest-neighbor search, or compressed attention keys, choose m from a target Kendall correlation and automatically expand the projection when measured ranking quality falls below the target.
Useful7/10
Difficulty3/10
Novelty7/10
✗ Mechanism failed
2026
Train decentralized agents using only individual rewards for discovering replenishable targets, while their observations contain conspecifics but not target locations. Give the policy a tunable visual or attention radius and test whether aggregation and improved search emerge above the predicted crossover, without adding alignment, proximity, or group rewards. This creates a controllable collective phase that can reduce redundant exploration and improve multi-agent resource discovery.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a quadratic pairwise attention or graph aggregation kernel with a compact, translation-invariant indefinite kernel approximated by signed random Fourier features. The feature map preserves the kernel's negative spectral mass through a diagonal sign matrix, so the resulting linear-time aggregation can represent similarities that ordinary positive-definite random features cannot.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Replace dense self-attention by a sparse mask on a one-dimensional or ordered token geometry, retaining local neighbors and adding long-range edges with probability proportional to distance raised to \(-(1+\sigma)\). Use \(\sigma\approx0.8\text{--}0.85\) as the initial regime because the paper finds that this range supports delocalized, GOE-like connectivity despite sparse bonds. The resulting layer has linear or near-linear attention cost while maintaining long-range paths.
Useful7/10
Difficulty4/10
Novelty6/10
✗ Mechanism failed
2026
Construct a sparse attention layer by sampling backward token histories as a continuous-time branching process rather than allowing every query to attend to every key. Each active ancestor either dies or branches into a bounded number of candidate ancestors, with branching probability controlled by a small parameter. The branch-out penalty predicts exponentially small probability of long, highly branching histories, providing a direct knob for receptive-field size and attention FLOPs.
Useful7/10
Difficulty5/10
Novelty7/10
Unverified
2026
Add a bank of temporal branches whose synapses apply heterogeneous delays before coincidence pooling. A branch selectively responds to an ordered pair or short sequence when the delay difference compensates the sequence interval, while low delay dispersion makes it primarily an order-invariant event detector. The branch threshold creates sparse, interpretable activations that can replace part of a recurrent or attention-based temporal module.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace an unconstrained attention or MoE routing matrix with a row-stochastic matrix that preserves a learned Gibbs-like distribution over groups. Anneal its temperature so that the router converges to a block-upper-triangular operator: mixing remains flexible within equal-energy groups, while cross-group traffic becomes directional rather than oscillating or collapsing. This creates a hierarchy-aware inductive bias and can reduce the number of active expert or attention connections.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace independent softmax routing or attention normalization with a differentiable approximate distribution over one-to-one assignments, using the Bethe permanent as the partition-function surrogate. Constrain the allowed token-to-expert or query-to-key support graph to have high girth, which gives an explicit bound on the approximation error and reduces short-cycle-induced correlations.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Use the paper's first-passage crossover as a controller for the communication or attention radius of a multi-agent policy. Instead of fixing a costly global attention range, estimate environmental depletion statistics online and set the radius near the predicted equality of individual and collective search times, expanding it only when the environment enters the collective-search regime. This turns a statistical-mechanical transition formula into an adaptive sparsification rule.
Useful6/10
Difficulty4/10
Novelty8/10
Unverified
2026
Replace flat expert logits with scores generated by Gaussian increments on a binary routing tree. A leaf receives the sum of increments on its root-to-leaf path, so sibling experts have correlated logits and the router can learn nested coarse-to-fine specialization; an inverse-temperature schedule controls the transition from exploratory diffuse routing to sparse routing.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace independent top-k token selection by a quality-weighted determinantal subset objective based on the Hardy–Szegő kernel. Tokens with high learned quality are preferred, but geometrically redundant tokens have a small determinant contribution, encouraging diverse sets of routed experts, retrieved items, or attended context tokens.
Useful6/10
Difficulty6/10
Novelty5/10
Unverified
2026
Replace or augment a dense attention similarity matrix with a Min-cone matrix generated by a monotone scalar sequence. The resulting matrix is positive semidefinite by construction, has only O(n) learned scalar parameters, and can be multiplied by values in O(n d) time using cumulative sums rather than forming an n-by-n matrix.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Use the number of tokens participating in an attention operation to adapt the projected query/key dimension according to the sharp Johnson–Lindenstrauss bound. Short attention windows receive a dimension limited by n-1, while longer windows use a logarithmic dimension, avoiding a fixed large head dimension when the current attention set contains too few points to require it.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace an all-pairs interaction or attention-like message field with an unbiased estimator obtained from a uniformly random partition into batches. Keep the partition fixed for \(\tau\) integration steps, then redraw it; the paper's error law predicts that too-long reuse causes a quadratic-in-\(\tau\) error accumulation, while finite-population error decreases as \(N^{-1}\).
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Use attention-graph hitting times to identify tokens whose information has not mixed through the network, then route only those tokens through additional Transformer blocks. Tokens with fast reachability exit early, while slow or isolated tokens receive more computation.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace dense attention or strictly local windows with a refreshed sparse communication graph whose edge lengths follow a truncated power law. For sequence positions or image patches at distance $r$, sample edges with probability proportional to $(1+r)^{-(d+\alpha)}$; in the regime $0<\alpha<d$, repeated layers should spread information across the sequence much faster than a local window while using only $O(nk)$ edges per layer. The main test is whether this preserves long-range accuracy at…
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Represent each Euclidean input point by its responses to randomly sampled half-spaces, then center those responses by the minibatch or source-distribution half-space occupancy. Use squared distances between these representations as a geometry-preserving substitute for raw Euclidean distances in graph-NN edge construction, cross-domain retrieval, or geometry-aware attention. The layer can compare point clouds from different ambient dimensions because each domain has its own half-space dictionary…
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Construct a graph on tokens using a cheap similarity or k-nearest-neighbor rule, partition it into small-radius pieces, and use one pooled state per piece for cross-piece communication. The paper's bound guarantees that the total number of vertices exposed by all one-hop piece neighborhoods is at most 4n, so the routing graph can have linear rather than quadratic communication budget. The approximation is controlled by increasing the number of within-piece basis summaries rather than making…
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Augment a causal sequence model with a small hierarchy of prefix summaries weighted by powers of the logarithmic rank of each preceding token. The summaries retain order-sensitive deviations from a baseline representation while costing O(KNd) for sequence length N, hierarchy width K, and hidden dimension d, instead of O(N^2d) dense attention.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Give each query-token pair a positive adaptive edge weight that evolves by a multiplicative rule instead of relying only on instantaneous dot-product attention logits. Edges whose aggregate interaction is useful can grow, while overloaded or incompatible neighborhoods can shrink. Sparse initialization is preserved because an edge initialized at zero remains zero under the multiplicative dynamics.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Add a learned scalar ordering to a directed graph attention layer and retain only forward edges, producing a DAG attention mask without requiring a supplied topological order. Train the ordering with a differentiable surrogate for weighted surplus, and regularize it toward the paper's explicit half-weight-minus-l2 certificate. This supplies a principled alternative to random masking or unconstrained bidirectional graph attention when causal or hierarchical information flow is desirable.
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Augment every graph or set token with a positive learned mass M_i that controls how strongly it contributes to other nodes and evolves through a growth-minus-inhibition equation. Use separate learned interaction kernels for state transport and mass inhibition, while retaining a directed interaction matrix so the layer is not forced to be permutation-symmetric or conservative.
Useful6/10
Difficulty5/10
Novelty6/10