△ Mechanism confirmed, baseline not beaten
2026
Replace a single scalar optimizer memory per parameter block with a small occupancy distribution whose bins represent distinct relaxation or gradient-history regimes. Train this state using a conservative redistribution operator and an energy-decreasing correction, allowing the optimizer to represent non-equilibrium lag and hysteresis that cannot be captured by one momentum variable.
Useful7/10
Difficulty6/10
Novelty8/10
✗ Failed on benchmark
2026
Use the reachability verifier as an optimization controller: permit a neural controller update only when the proposed parameter step remains inside a certified STL-safe trust region, and shrink the region when the reachable robustness margin collapses. This turns verification from an expensive final check into feedback that prevents gradient descent from crossing a temporal-logic feasibility boundary.
Useful7/10
Difficulty7/10
Novelty8/10
✓✓ Beats tuned baseline
2026
Replace the recurrent transition by a dissipative linear state update minus a maximal monotone nonlinear damping operator. Couple the hidden-state update to an output map so that the cell satisfies a discrete analogue of the paper's scattering-passivity inequality, controlling both hidden-state energy and output energy by initial-state energy plus input energy.
Useful7/10
Difficulty6/10
Novelty5/10
✗ Failed on benchmark
2026
Replace an unconstrained second-order residual or state-space block with a position-velocity system whose damping is the gradient or subgradient of a convex function. Compute the next state implicitly, so the damping cannot inject energy and the resulting layer is robust to large learned damping nonlinearities, nonsmooth activations, and long rollouts.
Useful7/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Add a safety projection step to every or selected Euler updates of a flow-matching action sampler. Instead of correcting only the first action, differentiate a collision-risk function through the predicted full action chunk, construct local linear inequality constraints, and apply the smallest correction that makes the future trajectory safe.
Useful7/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Attach a graph-Laplacian penalty to predictions on all labeled and unlabeled examples, with graph edges determined by augmentation-induced representation similarity. The supervised head is encouraged to vary smoothly along reliable augmentation edges, enabling labels to propagate through the unlabeled pool while preserving the paper's explicit augmentation-boundary diagnostic.
Useful7/10
Difficulty4/10
Novelty5/10
✗ Failed on benchmark
2026
Replace the fixed numerical stabilizer in signSGD by an exponentially decaying stability path, so the optimizer remains sign-like for a controllable duration instead of eventually reverting toward ordinary gradient descent as gradients become small. Sweep the decay rate as an explicit implicit-bias parameter: slower annealing should retain the non-Euclidean, barrier-like bias, while faster annealing should approach the sign endpoint more closely.
Useful7/10
Difficulty4/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Add conformal prediction sets for every action of a contextual policy, then select the action maximizing its worst-case utility over the corresponding set. Calibrate the sets using the outcome generated by this same max-min policy, rather than calibrating each action independently; this directly targets reliable utility under deployment decisions.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Construct a positive learning-rate schedule offline by minimizing the worst residual of every prefix on a normalized curvature interval, rather than optimizing only the final training horizon. The schedule is evaluated through the exact quadratic residual polynomial p_n(lambda) = product_{k=1}^n (1 - eta_k lambda), so every prefix is constrained to make progress across multiple curvatures.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace unconstrained final classifier prototypes with an equiangular tight frame (ETF), or initialize them as an ETF and softly preserve the structure during training. The frame gives every class the same norm, an isotropic aggregate geometry, and equal pairwise interference, which should improve conditioning and reduce class-prototype collapse in normalized-softmax or contrastive models. For arbitrary class counts where an exact ETF is unavailable, optimize differentiable tight-frame and…
Useful7/10
Difficulty4/10
Novelty5/10
✓ Mechanism works
2026
Search for a compact symbolic optimizer instead of selecting among fixed AdamW-like formulas. Encode optimizer programs as token sequences, learn a continuous variational representation of those sequences, and use a Gaussian-process Bayesian optimizer to propose promising update rules based on short neural-network training rollouts.
Useful7/10
Difficulty6/10
Novelty6/10
✗ Mechanism failed
2026
Train a two-dimensional neural deformation map with the paper's Lp conformal-distortion energy instead of using only a determinant or smoothness penalty. The resulting barrier penalizes near-folds and directional collapse while permitting useful nonrigid deformation, making it suitable for spatial transformers, image registration, and learned coordinate warps.
Useful7/10
Difficulty4/10
Novelty6/10
✗ Mechanism failed
2026
Replace ordinary momentum with a semi-implicit velocity update containing viscous damping and a proximal dry-friction step, while evaluating the gradient at a look-ahead parameter point. The dry-friction proximal operator exactly zeros sufficiently small velocities, which may suppress late-training oscillations and create finite-time stationarity instead of merely asymptotic velocity decay.
Useful7/10
Difficulty4/10
Novelty7/10
✗ Mechanism failed
2026
Replace an opaque MLP vector field with a stack of trainable symbolic primitives that can express linear terms, monomials, products, and related analytic operations. Apply an L1 penalty and prune small primitive coefficients after rollout training, yielding a compact dynamics module that is cheaper to evaluate and easier to inspect.
Useful7/10
Difficulty6/10
Novelty6/10
Unverified
2026
Treat a two-player neural game as a stochastic monotone operator problem and add an anchor regularizer that makes the current game strongly monotone during early training. Use stochastic extragradient steps inside geometrically decreasing regularization stages, retaining the accumulated anchor rather than resetting it; this should suppress rotational GAN dynamics while avoiding a permanent bias toward the initialization.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Use the paper's non-expansive spectral projection as a constraint-preserving optimizer step for trainable SPD matrices, including Mahalanobis metrics, covariance heads, graph kernels, and quantum density operators. After an SGD or AdamW proposal, symmetrize the matrix and clip every eigenvalue into \([\alpha,\beta]\).
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Add a nonnegative regret term for applying an action selected under a stale evaluator to the current evaluator. In a neural classifier, the action can be a prediction, routing decision, augmentation choice, or robust logit correction, while the evaluator is a vector of uncertainty or distribution-shift weights. The penalty is zero when the stale action remains optimal under the new evaluator and increases only when evaluator drift causes a value loss.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Represent a small ensemble of parameter vectors or a low-dimensional projection of parameters as a periodically driven probability distribution, and optimize the learning-rate/control waveform so that the distribution translates through successive optimization phases at approximately constant speed without unnecessary reshaping. The method adds a dissipation penalty based on probability-current mismatch and uses automatic differentiation through the particle or density dynamics to learn a…
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Replace unconstrained momentum updates for selected neural-network parameter blocks by damped second-order motion on a unit sphere. The update learns directions while preserving the block norm exactly, and includes the centripetal term required for a valid acceleration constrained to the sphere rather than merely projecting gradients after an unconstrained step.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a single recurrent hidden state by reciprocal forward and backward feature fields whose product defines a positive feature density. Feed the associated Bohm/Fisher potential back into both streams, creating adaptive curvature feedback that suppresses sharp incoherent relative structure and can localize the learned representation.
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Replace direct optimization of noisy simplex-valued mixture or MoE gate weights with a fixed number of EM responsibility updates. The finite iteration count acts as an implicit reverse-KL regularizer toward the uniform gate distribution, preserving low-frequency experts without selecting an arbitrary entropy coefficient.
Useful6/10
Difficulty4/10
Novelty4/10
Unverified
2026
Use the discrepancy between a learned potential and its long-time backward Lax–Oleinik evolution to identify dynamically critical states. Persistent near-contact points are candidates for the Aubry set and can guide adaptive collocation, while states with large gaps can receive fewer training samples.
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Use the determinant and trace-power identities of the rules matrix as a spectral diagnostic for recurrent or state-space training. Penalize unstable or excessively resonant modes through a truncated log-zeta objective, while retaining selected eigenvalues near the unit circle when long memory is desired. This gives a falsifiable transition criterion based on closed-walk growth rather than only gradient norms.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Apply the paper's empirical preimage-entropy construction to a learned recurrent transition map, penalizing excessive distinguishable hidden-state histories that produce the same current state while preserving multiple histories when the task requires genuine multimodality. Unlike a raw inverse-Jacobian penalty, the regularizer is computed only among inverse trajectories having similar empirical state distributions, so it distinguishes useful multimodal memory from uncontrolled branch explosion.
Useful6/10
Difficulty7/10
Novelty9/10