✗ Failed on benchmark
2026
Replace consecutive or randomly assigned transformed KV coefficients with groups whose variance-volume is approximately equal. Train one equal-size vector-quantizer codebook per group, so a fixed-width cache does not waste its low-rate budget by forcing high-variance and low-variance coordinates into badly mismatched groups. This is a drop-in quantization-layout change that can be applied to keys, values, or both.
Useful8/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace a conventional recurrent hidden state with a phase oscillator state whose stored memories are exponentially stable phase-locked configurations. Each memory has a coupling matrix or low-rank coupling parameter, while an external context selects which coupling landscape is active; this separates representation storage from sequence routing.
Useful8/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Group W consecutive diffusion or flow-model loss terms and approximate every intermediate parameter Jacobian by a time-weighted interpolation of the Jacobians at the group’s two endpoints. Sum the intermediate upstream signals into two endpoint cotangents, then perform only two full DiT backward passes instead of W. Add a cosine-similarity gate comparing predicted and actual intermediate velocity changes so that groups violating the local-linearity assumption use exact backpropagation.
Useful8/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace the usual random or elementwise-positive linear-attention feature map with a rank-one positive-semidefinite feature map derived from query and key vectors. For normalized inputs, the resulting kernel is the squared inner product, which is nonnegative and gives a geometrically structured interference pattern that is better suited to associative recall than an arbitrary low-rank feature map.
Useful8/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace an unconstrained linear recurrent or state-space memory with a finite-history recurrence whose coefficients are nonnegative and sum to one. The resulting companion transition is nonnegative and row-stochastic, guaranteeing spectral radius at most one while retaining a neutral constant-history mode at eigenvalue 1.
Useful8/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Add a non-autoregressive continuation layer to an RNN, SSM, or world model that predicts a future trajectory by solving for coefficients of a library of past trajectory windows and reusing those coefficients on the corresponding future windows. Unlike nearest-neighbor retrieval, the coefficients interpolate across multiple behaviors and can generalize to unseen systems whose output-visible eigenvalues are represented in the library.
Useful8/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Compress each hidden layer by retaining directions that are simultaneously reachable from the observed input distribution and observable at the network output. Unlike PCA or SVD, the retained subspace is weighted by downstream task sensitivity, so high-variance but output-irrelevant directions can be removed while low-variance predictive directions are preserved.
Useful8/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Train a low-dimensional projection of embeddings against centered pairwise geometry instead of only using raw-distance preservation or a JL-style guarantee. The loss removes the population or minibatch distance baseline before comparing distances, forcing the bottleneck to retain the fluctuations that carry ranking and task information.
Useful7/10
Difficulty4/10
Novelty6/10
✗ Failed on benchmark
2026
Replace random Fourier or random NTK features by a weighted deterministic quadrature rule for the kernel's feature integral. The resulting feature map uses the same linear-model interface as random features but can approximate the leading Gram-matrix eigenvalues substantially better at the same feature count, improving conditioning and reducing the width required for a target kernel approximation.
Useful7/10
Difficulty4/10
Novelty5/10
✗ Failed on benchmark
2026
Replace a full Hermitian curvature matrix, such as a Hessian or empirical Fisher matrix, by its block-diagonal version only when the paper's perturbation certificate predicts a small eigenvalue change. Use the certificate online to merge poorly separated blocks and retain independent preconditioners for well-separated blocks, yielding a controllable accuracy-memory tradeoff rather than a fixed block-diagonal approximation.
Useful7/10
Difficulty6/10
Novelty5/10
✗ Failed on benchmark
2026
Use the forward-backward reversal error as an online reliability signal: save more checkpoints or increase the low-rank dimension only when reversing a block produces a large defect. This turns the paper's observations about chaotic low-rank trajectories and rank deficiency into an adaptive memory-versus-gradient-accuracy controller.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace a conventional linear decoder in an autoencoder or latent state-space model with an explicit quadratic manifold decoder, allowing a small latent vector to represent curved and transport-like state trajectories. Add a dynamics-aware invariance loss that penalizes the discrepancy between the time derivative of the quadratic manifold and the neural dynamics evaluated on that manifold.
Useful7/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Replace a dense Haar or Gaussian random projection with a streamed product of random two-coordinate rotations followed by coordinate subsampling. The transform is exactly orthogonal before subsampling, requires only a list of rotation triples, and the paper's pseudo-mixing result predicts that degree-two statistics relevant to norm preservation and Johnson–Lindenstrauss embeddings become Haar-like after only O(n polylog(n)) rotations.
Useful7/10
Difficulty4/10
Novelty5/10
✗ Failed on benchmark
2026
Insert a fixed DPSS/prolate projection before an expensive neural block, retaining exactly the modes whose time-frequency concentration eigenvalues exceed a target threshold. Use the paper's tail-quantile formula to choose the projection rank from sequence length, effective bandwidth, and tolerated energy loss, then optionally learn a small correction in the retained coordinates. Unlike a Fourier truncation, the basis is optimized for simultaneous localization in the finite input window and the…
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace explicit zero-padding before FFT convolution by the paper's mixed-radix decomposition, which injects zeros through bounded tile sums and never allocates the padded input. The resulting transform is mathematically identical to the length-M transform of the explicitly padded signal, while reducing temporary storage and potentially memory bandwidth.
Useful7/10
Difficulty7/10
Novelty6/10
✗ Failed on benchmark
2026
Insert a linear Johnson–Lindenstrauss bottleneck around a set of jointly processed representations, choosing its width from the sharp finite-set dimension bound rather than from the model's nominal hidden size. The projection should preserve pairwise distances between tokens, patches, or retrieved items, allowing a downstream attention or MLP block to operate at lower width while retaining the geometry relevant to similarity computations.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Replace a wide collection of interchangeable near-zero branches with a module whose output is explicitly a quadratic form in the branch-weight Gram matrix. The module preserves the paper's leading-order behavior while making the relevant collective variable explicit and allowing low-rank parameterizations.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace blind width or rank expansion with a residualized candidate-selection procedure for low-rank adapters or expert banks. Each candidate block is measured after removing the component already represented by the incumbent network, filtered for numerical conditioning, and accepted only when its projected response produces sufficient loss-relevant gain. This should produce smaller adaptive networks than adding candidates by fixed order or magnitude heuristics.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace magnitude pruning in a trained recurrent network with stochastic pruning probabilities computed from weight magnitudes and the covariance of neuron activities under injected noise. Connections whose endpoints fluctuate in a sign-compatible way receive higher retention probability, while retained weights are rescaled to preserve average recurrent strength. The method uses local weights and activity covariance, avoiding Hessian construction and expensive global saliency optimization.
Useful7/10
Difficulty4/10
Novelty7/10
✗ Failed on benchmark
2026
Use the disorder-controlled stability boundary as a training schedule. Start with strong damping so optimization is well behaved, then reduce the damping margin toward zero to create long-lived oscillatory state memory after the network has learned useful representations.
Useful7/10
Difficulty4/10
Novelty7/10
✗ Failed on benchmark
2026
Replace a Markovian recurrent update with an MPS-valued temporal influence state that couples adjacent pairs of memory sites, mimicking the paper's CDU3 two-column construction. The hidden state retains structured correlations across multiple past time steps while computation remains linear in sequence length and polynomial in the bond dimension, rather than exponential in the memory horizon.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Split hidden dynamics into relaxation bands when the Jacobian spectrum has a gap, evolve each band with its own timescale, and retain an explicit cross-band exchange term. This yields a principled dual-timescale RNN or SSM rather than choosing fast and slow branches heuristically.
Useful7/10
Difficulty6/10
Novelty6/10
✗ Failed on benchmark
2026
Construct a neural activation bottleneck by projecting hidden states into a fixed covariance-eigenbasis and retaining only the d largest-magnitude coordinates per sample. For Gaussian, decorrelated activations, the paper proves that adaptive top-d selection in the PCA basis has no greater expected residual energy than adaptive top-d selection after any other orthogonal rotation. This provides a principled alternative to learning an unrestricted rotation before sparsification.
Useful7/10
Difficulty3/10
Novelty5/10
✗ Failed on benchmark
2026
Compress a transformer KV cache by selecting actual past tokens whose key or hidden-state columns form a stable basis for all cached tokens. Instead of retaining tokens with the largest attention scores or leverage scores independently, compute rank-revealing pivoting of the leading right-singular-vector matrix and retain its pivot columns, then evaluate attention using the representatives plus an optional low-cost residual correction.
Useful7/10
Difficulty5/10
Novelty6/10