✗ Failed on benchmark
2026
Replace a weight-tied residual or neural-ODE stepper with an explicit Runge–Kutta method satisfying the reused-last-stage conditions. The final derivative is evaluated at the exact endpoint and becomes the first derivative of the next step, saving one expensive neural-vector-field call per step while preserving the designed integration order.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Replace an opaque adaptive-optimizer state update with a small controller variable obtained by minimizing a strongly convex energy jointly associated with the proposed parameter motion. The controller is allowed to relax toward the current gradient before the parameter update, while the visible update uses the reduced energy and its envelope gradient. This creates an optimizer whose hidden geometry is optimized rather than inherited from a fixed exponential-moving-average recurrence.
Useful7/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Generate a family of multi-objective neural-network solutions by continuation rather than training each scalarization from scratch. Starting from one converged model, predict parameter changes as the constraint threshold moves, then apply a small number of Newton or quasi-Newton correction steps to recover a nearby Pareto-optimal model.
Useful7/10
Difficulty7/10
Novelty7/10
✗ Failed on benchmark
2026
Replace the fixed numerical stabilizer in signSGD by an exponentially decaying stability path, so the optimizer remains sign-like for a controllable duration instead of eventually reverting toward ordinary gradient descent as gradients become small. Sweep the decay rate as an explicit implicit-bias parameter: slower annealing should retain the non-Euclidean, barrier-like bias, while faster annealing should approach the sign endpoint more closely.
Useful7/10
Difficulty4/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Estimate an expensive fine-model trace or quadratic-form quantity using a telescoping sum over cheap-to-expensive neural approximations. Allocate many probes to cheap levels and only a few probes to the expensive level, exploiting strong correlation between adjacent levels to reduce variance at fixed compute. Candidate levels include truncated Transformer depth, reduced width, low-rank curvature, coarser graph resolution, or progressively tighter implicit-solver tolerances.
Useful7/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace independent Hutchinson vectors used to estimate traces of neural-network curvature operators with graph-coloring probing vectors. Coordinates that are far apart in an interaction graph share a color, so one probe simultaneously covers many coordinates while reducing variance from localized off-diagonal matrix entries. Apply this to Hessian-trace regularization, Fisher-trace diagnostics, or layerwise curvature estimates used by adaptive optimizers.
Useful7/10
Difficulty5/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Use the differentiable covariance chart to construct a Fisher-information preconditioner for the edge and innovation parameters of a linear-Gaussian neural module. Instead of applying an isotropic Euclidean update, whiten parameter steps according to how strongly they change the predicted Gaussian distribution. This targets ill-conditioning caused by redundant paths, correlated latent nodes, and badly scaled innovation covariances.
Useful7/10
Difficulty6/10
Novelty5/10
✗ Failed on benchmark
2026
Use the paper's parameterized invariant-torus residual and pseudo-arclength Newton correction to train a neural ODE across a continuous family of latent dynamical regimes. The continuation constraint allows the solver to pass through saddle-node folds, where stepping a physical control parameter alone would fail or jump to a different branch.
Useful7/10
Difficulty7/10
Novelty8/10
✗ Failed on benchmark
2026
Replace the sign-flip-only dynamics of high-index saddle search with low-rank inverse-curvature scaling on the estimated negative-curvature subspace. Directions with small negative Hessian eigenvalues then receive approximately curvature-independent updates instead of extremely slow updates proportional to their tiny curvature.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Build a recurrent or state-space layer whose transition matrix depends on a scalar pooled from the current hidden state. Estimate the local derivative of the scalar closure and penalize feedback gains that approach the fold threshold, preventing abrupt branch changes and excessive sensitivity.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Replace an unconstrained input-dependent multiplier on a recurrent fast-weight state with a sign-preserving tanh gate. The new state retains an additive low-rank update and optionally a separately modulated innovation term, but the accumulated-memory branch can never be amplified by a factor whose magnitude exceeds one.
Useful7/10
Difficulty4/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Construct a positive learning-rate schedule offline by minimizing the worst residual of every prefix on a normalized curvature interval, rather than optimizing only the final training horizon. The schedule is evaluated through the exact quadratic residual polynomial p_n(lambda) = product_{k=1}^n (1 - eta_k lambda), so every prefix is constrained to make progress across multiple curvatures.
Useful7/10
Difficulty5/10
Novelty6/10
✓ Mechanism works
2026
Search for a compact symbolic optimizer instead of selecting among fixed AdamW-like formulas. Encode optimizer programs as token sequences, learn a continuous variational representation of those sequences, and use a Gaussian-process Bayesian optimizer to propose promising update rules based on short neural-network training rollouts.
Useful7/10
Difficulty6/10
Novelty6/10
✓ Mechanism works
2026
Replace a single shared quadratic stability constraint in a routed state-space model with a path-complete family of quadratic certificates indexed by a small graph. During architecture search or training, identify bottleneck certificate nodes whose transition inequalities are nearly tight, split only those nodes, and re-solve the certificate problem. This should permit larger per-mode state transitions than a common Lyapunov matrix while retaining bounded hidden-state dynamics for arbitrary…
Useful7/10
Difficulty6/10
Novelty7/10
✓ Mechanism works
2026
Use the paper's inverse-temperature parameter to initialize networks containing m parallel depth-N branches. Choose branch count, depth, or an explicit aggregation scale so that beta = sqrt(2(N-1)/(n log m)) stays below the critical value sqrt(2), preventing the largest random branch from dominating the aggregate. This is applicable to residual multi-branch MLPs and other architectures whose block Jacobian is a sum of products.
Useful7/10
Difficulty4/10
Novelty7/10
✗ Mechanism failed
2026
Build a sequence or graph module whose state is a canonical set of affine tropical pieces rather than an opaque hidden vector. Compose consecutive segments by Minkowski addition of their lifted supports, merge alternatives by union followed by lower-hull reduction, and evaluate the resulting piecewise-linear function with a minimum.
Useful7/10
Difficulty7/10
Novelty8/10
✗ Mechanism failed
2026
Use an online estimate of the target distribution's remaining coordinate dependence to choose diffusion step sizes instead of using a uniform timestep grid. Take large jumps when the current state is nearly coordinate-factorized and shorten the step when the denoiser detects strong cross-coordinate dependence, targeting the paper's dependence-adaptive discretization scaling.
Useful7/10
Difficulty4/10
Novelty8/10
✗ Mechanism failed
2026
Replace conventional nested bilevel optimization with simultaneous primal-dual updates that enforce inner-model stationarity through a Lagrange multiplier. Add quadratic dual regularization and projection onto a bounded ball, while estimating all Hessian-vector terms using finite differences of ordinary gradients.
Useful7/10
Difficulty5/10
Novelty6/10
Audited (legacy)
2026
Replace independent per-action distributional value heads with a critic whose shared latent particle produces a vector of return samples for all actions simultaneously. Train the predicted joint return vector against a Bellman target vector formed from coupled counterfactual reward-transition samples, using a sliced Wasserstein loss. The greedy action is selected by the mean of the corresponding marginal particles, while shared particles retain cross-action dependence for learning and…
Useful7/10
Difficulty6/10
Novelty7/10
✗ Mechanism failed
2026
Represent Q-values using latent coefficients and a convex reconstruction operator rather than an unconstrained linear head. Enforce that reconstruction and compression are sup-norm nonexpansive, so the approximate Bellman operator remains a gamma-contraction and cannot exhibit the usual linear-function-approximation divergence.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Replace ordinary momentum with a semi-implicit velocity update containing viscous damping and a proximal dry-friction step, while evaluating the gradient at a look-ahead parameter point. The dry-friction proximal operator exactly zeros sufficiently small velocities, which may suppress late-training oscillations and create finite-time stationarity instead of merely asymptotic velocity decay.
Useful7/10
Difficulty4/10
Novelty7/10
✗ Mechanism failed
2026
Regularize a recurrent or state-space model using finite-time Lyapunov exponents of its actual hidden-state transition products. Penalize collapsed adjacent exponents while also controlling the largest exponent, encouraging several useful state directions instead of one dominant direction or universal contraction.
Useful7/10
Difficulty6/10
Novelty6/10
✗ Mechanism failed
2026
Replace an explicit Euler residual update for a skew-coupled hidden state with a five-stage palindromic composition of exact shear maps. Use a=1/4, the unique real coefficient maximizing the analyzed spectral CFL interval, and adapt the step size from an estimate of the learned coupling operator's spectral norm.
Useful7/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Use Adam normally, but periodically estimate the spectrum of the Adam-preconditioned Hessian and add a damped low-rank Newton correction when the preconditioned curvature is strongly ill-conditioned or the gradient is concentrated in flat directions. The correction is computed only in a small Lanczos subspace, so the method targets cross-coupled ill-conditioning without materializing or inverting the full Hessian.
Useful7/10
Difficulty6/10
Novelty6/10