✗ Failed on benchmark
2026
Replace a full-resolution global flow-matching or diffusion model with a hierarchy of local velocity fields operating on progressively finer grids. Coarse levels generate long-wavelength structure and pass it to fine levels through upsampling and residual conditioning, while every velocity network uses only a locality radius prescribed by the RG bound. This should preserve long-range correlations without quadratic global attention or a full-resolution global receptive field.
Useful8/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace the dense hidden-state trajectory of a continuous-depth or recurrent neural block by a rank-r factorization F(t) = X(t) S(t) V(t)^T, and evolve the factors with a reversible projector-splitting integrator. During backpropagation, reconstruct earlier hidden states by reversing the factor updates rather than storing all activations.
Useful8/10
Difficulty7/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Use the noisy diffusion state to estimate posterior probabilities over data clusters, then route the denoising computation to one or a few cluster-specific score experts. Each expert stores a low-rank covariance or low-rank score correction, so after posterior commitment the denoiser operates at the cluster intrinsic dimension rather than ambient dimension. During the early mixing phase, retain a shared full-dimensional score or evaluate a small weighted set of experts.
Useful8/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace unrolled autodiff through an ordered block-implicit neural layer with a custom reverse sweep that solves one small transposed local system per forward block update. The backward computes the exact gradient of the executed finite-depth solver while avoiding a global Jacobian and retaining only compact block information.
Useful8/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Replace vector-valued Hopfield neurons by SU(d)-valued latent states and construct Hebbian couplings from matrix memories. Recall is performed by iterating toward the dominant eigenmode of the induced lifted coupling operator, with each iterate projected back onto SU(d); the larger matrix representation should reduce random crosstalk and increase critical memory capacity.
Useful8/10
Difficulty7/10
Novelty8/10
✓✓ Beats tuned baseline
2026
Replace a full-resolution neural operator with an explicit multiscale reconstruction: a coarse predictor produces a low-resolution field, and a small corrector module combines its gradients and Hessians with learned rapidly varying basis functions. The model should recover fine detail without running the expensive backbone at fine resolution.
Useful8/10
Difficulty5/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Use a full primal-dual optimization solve in the forward pass, but backpropagate only through the last r iterations starting from a detached warm-start iterate. This avoids storing the full solver trajectory while preserving the forward solution, and provides a tunable bias-versus-memory tradeoff: r=0 is a cheap surrogate gradient, while increasing r should converge toward the implicit equilibrium gradient.
Useful8/10
Difficulty4/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Replace dense self-attention by a sparse attention graph whose neighborhoods satisfy the paper's size-dependent expansion condition. This preserves a logarithmically controlled route for every token subset to communicate with the rest of the sequence, reducing quadratic attention cost without allowing disconnected or poorly mixed token groups.
Useful8/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Use fast memory as read-only scratch state during the internal pondering iterations of a recurrent block, and apply memory writes only after the latent computation has halted or crossed a write gate. This prevents the transition operator from changing while it is being iterated, reducing self-corruption of the evidence used for subsequent reasoning.
Useful8/10
Difficulty4/10
Novelty6/10
✗ Failed on benchmark
2026
Replace the usual random or elementwise-positive linear-attention feature map with a rank-one positive-semidefinite feature map derived from query and key vectors. For normalized inputs, the resulting kernel is the squared inner product, which is nonnegative and gives a geometrically structured interference pattern that is better suited to associative recall than an arbitrary low-rank feature map.
Useful8/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace layer-local reconstruction in post-training quantization with a sequential objective that explicitly cancels the error already accumulated by the quantized prefix. For each layer, quantize its weights so that its local residual approximately negates the propagated incoming deviation, preserving the teacher trajectory even when the codebook is binary or 4-bit.
Useful8/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a dense directed attention matrix by a collection of K learned source-to-hub-to-target interactions. Each hub corresponds to a directed biclique, allowing many source tokens to communicate with many target tokens using O(NK) rather than O(N^2) pair interactions. The construction preserves asymmetric information flow and can be initialized from a graph cover of high-attention edges.
Useful8/10
Difficulty6/10
Novelty6/10
✗ Failed on benchmark
2026
Add a non-autoregressive continuation layer to an RNN, SSM, or world model that predicts a future trajectory by solving for coefficients of a library of past trajectory windows and reusing those coefficients on the corresponding future windows. Unlike nearest-neighbor retrieval, the coefficients interpolate across multiple behaviors and can generalize to unseen systems whose output-visible eigenvalues are represented in the library.
Useful8/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Compress a causal sequence by retaining history positions at equal increments of cumulative representation variation instead of at uniform time intervals. Use the resulting N representatives in a decoder that reconstructs piecewise-constant keys and values; the paper's minimax result predicts a worst-case reconstruction error of total variation divided by 2N, independent of where rapid changes occur.
Useful8/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace token-by-token KV storage after an SSM or recurrent encoder with an online allocate-on-novelty cache. A new slot is created only when the incoming key is sufficiently dissimilar from every stored key; otherwise the incoming value is merged into its nearest slot, so repeated or redundant content does not grow the cache.
Useful8/10
Difficulty4/10
Novelty6/10
✗ Mechanism failed
2026
Partition a large graph into induced subgraphs and perform most parameter updates using only local subgraphs, interleaving them with inexpensive global updates on a randomly subsampled coarse graph. The coarse correction preserves information about cross-partition dependencies while reducing full-graph message passing and communication cost.
Useful8/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Compress a trained wide analytic-activation MLP by fitting a narrow same-depth student to the teacher's function values and input derivatives, rather than matching only outputs on a calibration dataset. Choose the student width from the input dimension and target error, with a target scaling m = O((log(1/epsilon))^d_in), and use sequential layer fitting plus channel reweighting to limit error accumulation through depth.
Useful8/10
Difficulty6/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Give every possible pairwise relation its own recurrent latent state, stored in a dictionary keyed by stable node identities, instead of discarding the state whenever the edge is absent from the current graph. At each timestep, active edges retrieve their old state, update it with the current pair features and neighborhood messages, and write it back after message passing. This should improve predictions in dynamic graphs with intermittent interactions and reduce the burden on node states to…
Useful7/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Use a spherical-design codebook and the paper's polar slack factorization to create a nonnegative geometric interaction bias for attention or expert routing. The resulting kernel is generated by a rank-one term and a rank-at-most-d term, and entries close to zero can define a structured sparse mask instead of relying only on learned top-k selection.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace dense or single-dilation sparse attention with two sequential sparse attention stages whose offsets form co-prime arithmetic progressions. The first stage mixes tokens separated by multiples of M2, the second by multiples of M1; their composition reaches virtual offsets mM2+nM1, providing many structured long-range interactions from only M1+M2-1 physical offset families. Use causal masking and residual connections so the module can replace a standard transformer attention block without…
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Support conjunction queries over multiple roles without explicitly storing a huge tensor of repeated objects. Represent the required higher-order memory through query-dependent contractions, enabling compositional retrieval with memory that scales linearly in the number of objects.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Compress a dataset for a frozen neural embedding and multi-output linear head into at most (m+1)r weighted examples while preserving the full-data minimum-Frobenius-norm least-squares solution. This can make repeated linear probing, classifier refitting, or replay-buffer training much smaller without changing the optimum in exact arithmetic.
Useful7/10
Difficulty6/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Constrain recurrent preactivations to remain nonnegative so that ReLU acts as the identity along realized trajectories. The hidden dynamics then admit a classical linear observability matrix, allowing principled hidden-coordinate selection and conditioning control instead of relying on potentially destructive activation masks.
Useful7/10
Difficulty5/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Use a cyclic forward-neighbor recurrent or state-space layer and regularize its coupling so selected discrete Fourier modes are contracting while task-critical modes remain weakly damped. The paper's exact mode factors make instability falsifiable: a mode becomes unstable when its scalar factor changes sign, producing a measurable transition rather than a vague smoothness prior.
Useful7/10
Difficulty5/10
Novelty8/10