△ Mechanism confirmed, baseline not beaten
2026
Allocate different entropy budgets to different KV-cache blocks instead of assigning every token and head the same nominal bitwidth. Use the ECASQ Lagrangian so high-variance or attention-sensitive blocks receive more codepoints, while predictable blocks collapse to fewer symbols and become highly compressible. Preserve unbiasedness per scalar or block so reconstructed keys and values have zero mean quantization error conditional on the original tensor.
Useful8/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace fixed-grid stochastic quantization of each tensor block with an adaptive ordered codebook selected under both an entropy budget and a maximum number of codepoints. Within every interval between adjacent codepoints, use unbiased stochastic interpolation, so the quantized block remains unbiased while the emitted symbol distribution becomes easier for arithmetic or Huffman coding to compress. The representation should reduce actual compressed bytes at fixed MSE, or reduce MSE at fixed…
Useful8/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Turn an iterative optimization or equilibrium computation inside a neural network into a differentiable layer whose backward pass solves the implicit adjoint system with conjugate gradients or GMRES using only automatic-differentiation matrix-vector products. This avoids storing unrolled iterations and avoids explicit Hessian or Jacobian construction, enabling longer solver horizons and lower-memory implicit architectures.
Useful8/10
Difficulty6/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Before quantizing a matrix product, reparameterize its factors as A'=AT and B'=T^{-1}B, preserving the exact full-precision product while changing the quantization difficulty of each factor. Choose a positive diagonal T=diag(t_1,...,t_K) that minimizes predicted post-quantization product error, rather than using output-channel scaling or a fixed heuristic grid. The gauge can be shared across several products when transformed-copy cost matters.
Useful8/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Implement causal linear attention in chunks and combine chunk summaries with an associative scan instead of carrying the recurrent state through all chunks sequentially. This preserves the exact causal computation while reducing inter-chunk dependency depth from the number of chunks to its logarithm, enabling substantially more GPU parallelism for long-context training and prefill.
Useful8/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace layer-local reconstruction in post-training quantization with a sequential objective that explicitly cancels the error already accumulated by the quantized prefix. For each layer, quantize its weights so that its local residual approximately negates the propagated incoming deviation, preserving the teacher trajectory even when the codebook is binary or 4-bit.
Useful8/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Compress each layer's KV tensor with a partial Tucker approximation over token and feature axes, then encode the truncation residual with a rotated uniform quantizer. Select token rank, feature rank, and residual bit-width jointly under a global byte budget, allowing values with flat spectra to receive residual bits while keys may receive more low-rank capacity.
Useful8/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace Euclidean or entrywise Kronecker fitting of a layer curvature matrix with its affine-invariant projection onto G = A tensor B. Use the resulting factors as a compact SPD preconditioner in the optimizer, while solving the projection through logarithmic residual partial traces and Armijo line search.
Useful8/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace MSE-calibrated scalar quantization with a conservative left-edge quantizer whose thresholds are denser where activation probability and task utility slope are both high. For a monotone utility function, this should preserve high-impact activation regions better than uniform or MSE-optimal bins at the same number of codes, while retaining an explicit rate-versus-quality design rule.
Useful7/10
Difficulty4/10
Novelty7/10
✗ Mechanism failed
2026
Replace ordinary token merging or graph pooling with a learned block map whose output preserves information about a remote target conditioned on the surrounding coarse representation. The paper's majority-spin counterexample gives a concrete failure mode: two microscopic configurations mapped to the same pooled token can imply different predictions for distant variables.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace flat all-pairs attention with attention neighborhoods induced by a compatible tree over tokens, patches, nodes, or retrieved items. Retain exact or approximate attention inside nearby tree subtrees and add a path-monotonicity regularizer so semantically distant endpoints are not more similar than intervening tree neighbors.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace full PSD self-attention with a pivoted Cholesky/Nyström approximation whose landmarks are sampled from the unexplained diagonal mass. Tokens with large residual self-similarity are more likely to become landmarks, so the rank budget is spent on difficult regions rather than uniformly selected tokens.
Useful7/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a fixed-size learned latent for an array-valued complex tensor with a variable-length list of continuous rank-one spectral atoms. An encoder predicts candidate receive direction, transmit direction, residual off-grid offsets, and complex gains; the decoder reconstructs the tensor analytically from the array-response formula, so changing the antenna dimensions does not require changing the decoder weights.
Useful7/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Replace fixed chunked-prefill settings in an LLM serving engine with a feedback controller that decreases the number of prompt tokens processed per scheduling quantum as GPU saturation or long-context load increases. The controller targets a high-quantile bound on the absolute GPU-power ramp while preserving the existing peak-power ceiling and measuring the resulting latency-throughput tradeoff.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Parameterize a trainable weight update as \(\Delta W=UV^{\top}\) with an excessive initial rank \(r\), and penalize active columns using an exact column \(\ell_{2,0}\) penalty. Increase \(\lambda\) along a warm-started path and hard-delete redundant paired columns, producing an automatically selected rank without training a separate model for every candidate rank. Apply scale balancing after each update so pruning decisions are invariant to reciprocal rescaling of factor pairs.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a portion of quadratic key-value attention or an external episodic table with a per-sample matrix fast memory updated by rank-one delta corrections. The memory directly learns a linear key-to-value map and can be carried across sequence segments, providing cheap online adaptation with constant state size per head.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Replace deterministic LoRA importance scores with one-sided tests of whether each rank-one update has population contribution at least a user-selected threshold. Maintain empirical contribution samples during fine-tuning, estimate their uncertainty, and prune the components with the weakest statistical evidence while respecting the target rank budget. The method should avoid deleting components merely because their latest minibatch gradient was small.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace dense query-key attention with an adaptive cross approximation constructed from selected query and key pivot tokens. At each rank, choose the pivot pair that removes large estimated residual energy, update the residual by a rank-1 cross correction, and stop when the residual estimate reaches a target tolerance. The resulting factorization computes approximate attention using a small number of landmark interactions while adapting to the actual token distribution.
Useful7/10
Difficulty6/10
Novelty5/10
✗ Mechanism failed
2026
Use the activation singular-value spectrum to allocate a fixed zeroth-order parameter budget across layers instead of assigning the same rank everywhere. Layers with a large discarded singular-value tail receive more coefficient directions, while spectrally compressible layers use smaller adapters, preserving the gradient-relevant subspace under a global memory and query budget.
Useful7/10
Difficulty4/10
Novelty6/10
Audited (legacy)
2026
Replace uniform embedding dimensions with a globally budgeted allocation based on the estimated spectral complexity of each categorical feature. Tables whose category representations have large leading singular-value energy receive more dimensions, while high-cardinality tables are penalized because each extra dimension consumes more parameters.
Useful7/10
Difficulty4/10
Novelty6/10
Unverified
2026
Construct each hierarchy transition as a coarse-graining map together with a local decoder that recovers the retained local representation. This creates a reversible-information path for features needed by later computation while still allowing high-frequency or locally irrelevant components to be discarded.
Useful6/10
Difficulty6/10
Novelty5/10
Unverified
2026
Replace unconstrained token-to-expert routing with a nonnegative mixture of a small set of feasible routing or communication patterns. Enforce resource limits using the paper's one-sided positive-contribution bound, which gives a conservative certificate without enumerating all joint token activation scenarios. Expand the pattern set only when a separation procedure finds a routing pattern that improves the router objective while adding useful capacity information.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Add a recurrent associative matrix to each selected transformer layer so recent key-value relationships can be retrieved without retaining every past token or performing gradient updates. The matrix uses input-dependent retention and write gates, but retrieval is always performed from the pre-write state, preventing the current target from leaking into its own prediction. Frobenius-norm clipping makes the recurrent memory bounded and provides a direct stability control.
Useful6/10
Difficulty4/10
Novelty4/10
Unverified
2026
Replace the dense output of selected linear projections with a two-sided magnitude threshold that emits zero for small values but preserves signed large values. Learn one positive threshold per projection, or optionally one threshold per output channel, so the network discovers where sparse events can be removed while retaining outlier information.
Useful6/10
Difficulty4/10
Novelty5/10