✓✓ Beats tuned baseline
2026
Compress only the difference between the current client gradient and a persistent control variate, rather than compressing the full gradient. As the control variate tracks the client gradient, the residual shrinks and the same communication budget produces less compression noise than direct gradient quantization.
Useful7/10
Difficulty4/10
Novelty5/10
✗ Failed on benchmark
2026
Replace the single global MoE capacity factor with expert-specific capacity reservations chosen from a small reliability menu. Experts with highly variable or operationally important token loads receive larger robust buffers, while predictable experts run closer to their mean load. This should reduce token dropping and padding waste simultaneously, especially under distribution shift or bursty routing.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Train an admission or MoE routing gate not only to reduce its immediate workload, but also to preserve the ordering between a controlled trajectory and a baseline trajectory under the same request stream. Penalize counterfactual events in which the controlled system, after initially rejecting work, later exceeds the baseline workload because its changed state causes a large job to be admitted.
Useful7/10
Difficulty5/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Replace uniform node minibatches in a GNN with a coreset selected from a small random candidate set using local Laplacian-column coherence. Select nodes whose connectivity signatures are least redundant with already selected nodes, while retaining inverse-probability weights for unbiased loss estimates. This should improve coverage of weakly connected graph clusters and preserve smooth graph signals at the same batch size.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace ordinary graph convolution with a Markov diffusion whose edge transition probability is proportional to a geometric distance weight d(v,u)^(n−2), following the paper's Delaunay discretization theorem. Use graph Dirichlet capacity on expanding node shells to detect whether information escapes through the graph or repeatedly returns to local neighborhoods, and use this signal to select propagation depth or add a regularizer.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a soft auxiliary load-balancing loss with a deterministic router correction that balances expert assignments separately within confidence or score shells. The correction preserves the number of tokens assigned to each shell while making every expert receive equal or nearly equal mass inside each shell, preventing high-confidence tokens from monopolizing a subset of experts.
Useful7/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace backpropagation through a small encoder with an online local update driven by consecutive examples and a fixed random projection of hidden activity. The projection produces a modulatory signal that encourages temporally adjacent inputs to have compatible representations, while the homeostatic term prevents sigmoid units from saturating or collapsing.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace synchronous replicated-gradient computation with a bounded-staleness stream: at optimizer step t, aggregate one gradient for each data partition, using the newest completed evaluation even if it was computed at an earlier model version. Replicated partition placement makes the aggregate robust to stragglers, while pipelining ensures that each worker computes only one partition gradient per step.
Useful7/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace node-by-node scheduling in an iterative message-passing network with a learned scheduler that selects one graph cluster at a time, while updating all nodes in that cluster synchronously. The scheduler observes a quantized histogram of local residual weights, making its state invariant to permutations of nodes inside a cluster and independent of cluster cardinality.
Useful7/10
Difficulty5/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Train one denoiser only for the nonquadratic residual distribution, then modify the diffusion sampler using an analytically computed quadratic Gaussian context. Changing $K$ at inference changes the target distribution without retraining the denoiser, enabling transfer across temperatures, masses, coupling strengths, and boundary conditions whenever those changes remain quadratic.
Useful7/10
Difficulty6/10
Novelty6/10
✗ Failed on benchmark
2026
Replace a fixed distributed gradient-reduction topology with a topology selected from recent estimates of each shard's mean and variance. The selector minimizes the paper's predicted second-moment rounding cost, favoring shallow placement for high-variance coordinates and Huffman-like placement for unequal-variance blocks, while retaining the same number of additions and communication volume.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Mechanism failed
2026
Replace greedy uncertainty sampling with a shallow Monte Carlo Tree Search that plans sequences of neural-network data acquisitions using a propagated uncertainty state. Each hypothetical query reduces uncertainty at nearby or correlated points, so later rewards automatically penalize redundant coverage and include labeling, simulation, or trajectory-transition costs.
Useful7/10
Difficulty6/10
Novelty6/10
✗ Mechanism failed
2026
Use the KS ratio to decide how many message-passing layers to execute per graph or per node, rather than selecting a fixed depth. In the subcritical regime, stop once the predicted remaining effect is below a tolerance; in the supercritical regime, continue until the observed logit change becomes small or a larger budget is reached.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Construct a neural activation bottleneck by projecting hidden states into a fixed covariance-eigenbasis and retaining only the d largest-magnitude coordinates per sample. For Gaussian, decorrelated activations, the paper proves that adaptive top-d selection in the PCA basis has no greater expected residual energy than adaptive top-d selection after any other orthogonal rotation. This provides a principled alternative to learning an unrestricted rotation before sparsification.
Useful7/10
Difficulty3/10
Novelty5/10
✓✓ Beats tuned baseline
2026
Treat unresolved inference items as active threats and allocate a fixed budget of C module evaluations per round. Each evaluation has an item-dependent probability of completing the item, while the scheduler observes only completion or failure after the round. Use fair allocation when completion probabilities are unknown or nearly homogeneous, then switch to a marginal-success greedy policy as feedback estimates become reliable.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Replace dense self-attention by a sparse mask on a one-dimensional or ordered token geometry, retaining local neighbors and adding long-range edges with probability proportional to distance raised to \(-(1+\sigma)\). Use \(\sigma\approx0.8\text{--}0.85\) as the initial regime because the paper finds that this range supports delocalized, GOE-like connectivity despite sparse bonds. The resulting layer has linear or near-linear attention cost while maintaining long-range paths.
Useful7/10
Difficulty4/10
Novelty6/10
✗ Failed on benchmark
2026
Construct a finite-range, translation-equivariant recurrent convolutional module with an absorbing inactive state, then train its local dynamics so that seeded activity crosses coarse-grained space-time blocks with probability above an oriented-percolation threshold. This should produce reliable long-range propagation without dense global attention while remaining robust to non-monotone local updates and perturbations. Block statistics also provide a diagnostic for vanishing propagation or…
Useful7/10
Difficulty5/10
Novelty8/10
△ Mechanism confirmed, baseline not beaten
2026
Use the paper's finite-width O(n^{-1/2}) Gaussian-process approximation bound as a width-budgeting rule rather than choosing every hidden dimension uniformly. Estimate an architecture-specific constant for each layer or attention contraction, then allocate width according to the smallest dimension satisfying its allowed distributional error. This should produce narrower models at comparable GP-like behavior, or permit the same parameter budget to be concentrated in the layers where finite-width…
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Estimate an expensive fine-model trace or quadratic-form quantity using a telescoping sum over cheap-to-expensive neural approximations. Allocate many probes to cheap levels and only a few probes to the expensive level, exploiting strong correlation between adjacent levels to reduce variance at fixed compute. Candidate levels include truncated Transformer depth, reduced width, low-rank curvature, coarser graph resolution, or progressively tighter implicit-solver tolerances.
Useful7/10
Difficulty6/10
Novelty6/10
✗ Mechanism failed
2026
Construct a sparse attention layer by sampling backward token histories as a continuous-time branching process rather than allowing every query to attend to every key. Each active ancestor either dies or branches into a bounded number of candidate ancestors, with branching probability controlled by a small parameter. The branch-out penalty predicts exponentially small probability of long, highly branching histories, providing a direct knob for receptive-field size and attention FLOPs.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Construct a sparse radius graph over input samples or learned node embeddings, compute its cycle space, and remove the subspace generated by sufficiently short cycles. Feed the remaining quotient-cycle coordinates or Betti-rank estimate to a graph neural network as a global topological feature, or use them to guide pooling so that local redundant loops are collapsed while global loops are retained. The paper predicts that the threshold L approximately equal to |log r| graph hops is the critical…
Useful7/10
Difficulty6/10
Novelty7/10
Unverified
2026
Add a bank of temporal branches whose synapses apply heterogeneous delays before coincidence pooling. A branch selectively responds to an ordered pair or short sequence when the delay difference compensates the sequence interval, while low delay dispersion makes it primarily an order-invariant event detector. The branch threshold creates sparse, interpretable activations that can replace part of a recurrent or attention-based temporal module.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace an unconstrained attention or MoE routing matrix with a row-stochastic matrix that preserves a learned Gibbs-like distribution over groups. Anneal its temperature so that the router converges to a block-upper-triangular operator: mixing remains flexible within equal-energy groups, while cross-group traffic becomes directional rather than oscillating or collapsing. This creates a hierarchy-aware inductive bias and can reduce the number of active expert or attention connections.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace a finite-dimensional latent prior for fields or trajectories with a continuous flow acting on function coefficients in a separable Hilbert space. A Gaussian reference with an explicit covariance spectrum is transformed by a learned vector field, so the model can represent smooth, non-isotropic function distributions while sharing parameters across discretization resolutions.
Useful6/10
Difficulty6/10
Novelty6/10