✗ Failed on benchmark
2026
Replace a full-resolution global flow-matching or diffusion model with a hierarchy of local velocity fields operating on progressively finer grids. Coarse levels generate long-wavelength structure and pass it to fine levels through upsampling and residual conditioning, while every velocity network uses only a locality radius prescribed by the RG bound. This should preserve long-range correlations without quadratic global attention or a full-resolution global receptive field.
Useful8/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Train a neural trial function for an elliptic PDE using a very-weak residual in which all derivatives act on fixed smooth test functions rather than on the neural network. This eliminates second-order reverse-mode or forward-mode automatic differentiation and allows low-regularity activations while retaining a least-squares objective over many test functions.
Useful8/10
Difficulty4/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace full-dimensional node or weight perturbation with perturbations in an input-conditioned d-dimensional tangent subspace, where d is the input or feature dimension and is much smaller than the reservoir width or parameter count. Estimate the update using only scalar self-supervised losses from positive and negative perturbations, then map the low-dimensional update back to the trainable parameters.
Useful8/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Replace the standard squared pointwise PDE residual in an elliptic PINN by its discrete $H^{-1}$ norm. The residual is passed through an inverse Dirichlet Laplacian, reducing the dominance of rapidly varying residual modes and acting as a mathematically specified preconditioner for the PINN training gradients.
Useful8/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Add a fading-memory consensus force to parameter dynamics, pulling the current parameter toward a distributed average of its past while preserving the ordinary gradient step. Implement the infinite memory through one or several recursive exponential states, and tune the memory decay so that quadratic-mode dynamics remain exponentially stable. This should suppress oscillations and catastrophic steps without relying on conventional momentum alone.
Useful7/10
Difficulty4/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace isotropic Langevin noise in latent or energy-based neural sampling with a smooth position-dependent temperature \(\sigma(x)\geq 1\). Use the divergence correction associated with the diffusion matrix so that increasing exploration in the tails does not change the desired target distribution. This should reduce metastability and improve effective samples per gradient evaluation on heavy-tailed latent posteriors.
Useful7/10
Difficulty4/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Train a low-width network by repeatedly selecting a normalized neuron that is maximally correlated with the current residual, then refit all output coefficients jointly. This gives a constructive alternative to random initialization of all hidden units and exposes an empirical width-versus-error curve that can guide early stopping or architecture selection.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Build a label-free router for a finite library of neural operators by estimating one shared physical target response from an anchor prediction and using it to rank every candidate through inner products with candidate differences. The method avoids running a full residual-based diagnostic independently for every expert and can be used either to select the best expert or to form a corrected weighted combination.
Useful7/10
Difficulty6/10
Novelty8/10
✗ Mechanism failed
2026
Replace a trainable shallow MLP hidden layer by a frozen bank of smooth sigmoid ridge functions and train only a linear output head. Choose the feature count and parameter sampling regime using the theorem's explicit dependence on input dimension d, target regularity k, evaluation norm m, and confidence delta. The construction is especially appropriate for smooth regression, scientific surrogate models, and PINNs, where derivatives of the network output are part of the loss.
Useful7/10
Difficulty3/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Build a neural operator around explicit input and output measurement spaces rather than forcing the network to consume and emit a fixed grid. The same learned latent surrogate can be reused on alternative sensor layouts or query meshes through reconstruction and re-encoding maps, with a consistency loss enforcing agreement between measurement pipelines.
Useful7/10
Difficulty6/10
Novelty5/10
✗ Failed on benchmark
2026
Replace the naive pseudospectral evaluation of a quadratic neural-operator nonlinearity with a two-point split-form product. Use the entropy-stable (alpha, beta) = (1/3, 2/3) split as the default, or learn alpha under the consistency constraint alpha + beta = 1 while monitoring energy growth. The goal is to suppress weakly underresolved aliasing and prevent long-horizon rollout blow-up without full 2/3-rule zero-padding.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace the standard diagonal or identity preconditioner used when solving an implicit neural layer with a coarse/fine Schur-complement preconditioner. The hidden state is decomposed into a low-dimensional coarse subspace and its orthogonal complement; the coarse interaction is solved accurately, while the fine block receives a damped approximate inverse. The method is especially suitable for deep equilibrium models, implicit MLPs, and Newton or quasi-Newton training of residual dynamics.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a stack of local message-passing layers by a fractional spectral graph filter implemented through a small bank of sparse shifted Laplacian solves. The fractional exponent controls how strongly the layer mixes information across graph distances, while rational approximation avoids dense eigendecomposition and supports efficient differentiation through iterative linear solvers.
Useful7/10
Difficulty6/10
Novelty5/10
✓✓ Beats tuned baseline
2026
Replace a dense mixing or attention matrix on tokens arranged on a Cartesian grid by a product of learned or fixed one-dimensional concentration operators. The layer applies one axis operator at a time, reducing parameter and compute cost while enforcing that the global operator is a positive contraction with controlled spectral leakage.
Useful7/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace dense attention between tokens on opposite sides of a one-dimensional boundary or segment split with a dyadic low-rank approximation of a Cauchy/Hankel distance kernel. Each distance-scale block uses O(log(1/\varepsilon)) features, and the number of active scales grows only logarithmically with context length after discarding a narrow boundary layer. This is especially suitable for a relative-position attention branch or state-space-like long-range branch, rather than arbitrary…
Useful7/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Combine the very-weak residual with step activations and one-bit weights, so the deployed PDE solver uses threshold and binary operations while training still optimizes a differentiable surrogate. The weak objective only needs values of the trial function and therefore does not require differentiating discontinuous activations with respect to spatial coordinates.
Useful7/10
Difficulty6/10
Novelty4/10
✗ Failed on benchmark
2026
Reparameterize a recurrent or state-space layer so that its hidden-state update contains an explicit stabilizing feedback controller, while the neural network learns only a residual control in the feedback coordinates. Choose K to reduce finite-horizon state-propagation amplification, suppressing exploding hidden states and gradients on long sequences.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Mechanism failed
2026
Replace a dense coordinate-kernel interaction among N points by an orthogonal samplet transform with a sparse detail-detail matrix and a small polynomial branch. Detail basis vectors have vanishing moments, so smooth low-frequency behavior is represented by a few polynomial coefficients while localized residual interactions become sparse in the transformed domain.
Useful7/10
Difficulty6/10
Novelty7/10
Unverified
2026
Use two bounded self-adjoint transition operators in a recurrent or state-space block and penalize their normalized Hilbert--Schmidt commutator. When the penalty is small, the paper guarantees that the pair is close to exactly commuting operators, suggesting a controlled path to a shared eigenbasis and cheaper coordinate-wise dynamics. Add an optional numerical repair step that projects the learned pair toward a simultaneously diagonalizable pair.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Model dynamic routing as a multitype branching process: an active token of type d probabilistically creates child activations of type d'. Estimate the corresponding mean offspring operator and regulate its Perron root to a target reproduction rate, typically near one. This should make adaptive-depth or recursively routed networks use sparse computation without producing either rapidly vanishing paths or uncontrolled activation explosions.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a trainable shallow hidden layer by a deterministic feature dictionary generated from Chebyshev-spaced scalar parameters and quasi-uniform ridge directions. Train only the output linear map, or use the frozen layer as the first stage of a larger network, thereby eliminating hidden-layer backpropagation while retaining a constructive smooth-function approximation guarantee.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Replace projected overdamped Langevin updates for constrained neural-network parameters with underdamped Langevin dynamics carrying an explicit momentum variable and specular reflection at the boundary of a convex parameter domain. The paper's hypocoercive result predicts a convergence rate proportional to the square root of the Poincare constant of the target position distribution, potentially giving substantially faster mixing in poorly conditioned constrained problems than overdamped…
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Use Halpern iteration to solve a non-expansive neural equilibrium layer from temporally correlated samples, and estimate its stochastic operator with a PAGE-style refresh/difference estimator. The anchor supplies a vanishing but explicit stabilizing force, while same-state differences reuse consecutive Markov samples and should reduce the number of full oracle evaluations required for a target fixed-point residual.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace a global attention truncation rule with a per-query local-tail budget. For each query token, retain nearby or high-priority keys until the estimated discarded interaction strength is below a target epsilon; this uses the paper's central distinction between local tail mass and the extensive norm of the discarded operator. The resulting attention pattern can allocate long-range computation only to tokens whose local tail is large.
Useful6/10
Difficulty5/10
Novelty5/10