Unverified
2026
Replace the purely diagonal preconditioner in AdamW or SGD with a blockwise, single-secant BFGS inverse-curvature metric. Use spectral damping and clipping relative to the diagonal RMS metric so the learned metric cannot become arbitrarily ill-conditioned, mirroring the paper's uniform comparison between its conjugate-free scaling and the primal barrier Hessian.
Useful6/10
Difficulty6/10
Novelty5/10
Unverified
2026
Replace the ordinary update in a differentiable minimax game with a Halpern-anchored second-order operator step. The current game iterate is first corrected using the local Jacobian of the game gradient, and the corrected point is then contracted toward a fixed anchor with a decreasing Halpern weight. This is intended to reduce cycling in adversarial training while preserving the faster asymptotic behavior associated with second-order monotone-operator methods.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace the scalar learning rate of SGD or Adam's outer update by a blockwise Barzilai--Borwein estimate computed from consecutive parameters and gradients. Use gradient smoothing, denominator checks, and clipping so that the curvature estimate remains usable with stochastic neural-network gradients.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Use the paper's stationarity-floor scale to set the SAM radius from a desired gradient tolerance, and reduce the radius when training approaches that tolerance. This turns an otherwise opaque SAM hyperparameter into a curvature- and accuracy-aware schedule.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Replace fixed-period federated averaging or distributed all-reduce with a Bernoulli communication decision whose probability is selected from estimated network connectivity and optimization conditioning. Local workers continue making corrected updates between communication events, while the contraction theorem exposes when communication is worth its cost.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Represent stochastic layer execution, branching, retries, and early exit as a finite continuous-time Markov chain, with the completed-prediction state absorbing. Learn transition rates jointly with neural-network weights, but use MFPT sensitivities to allocate rate changes according to their available control budget rather than allowing one routing edge to dominate halting-time control.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Use the Bregman objective's exact residual-dependent curvature to build a positive-semidefinite Gauss-Newton preconditioner for a neural network's scalar regression head. Negative curvature weights are clipped or damped before solving the update, preserving the original gradient while preventing residual patterns from producing unstable parameter steps.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace a fixed-batch SGD or proximal-gradient update by a stochastic proximal-subgradient step whose step size is backtracked against an empirical sufficient-decrease condition. If the condition is too noisy or repeatedly fails, enlarge the batch and retry; otherwise retain the current batch, allowing sample size to grow only when needed.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace uniform or purely loss-driven update allocation with a scheduler that targets both the mean update rate and the temporal variance of updates for each parameter group, task, or expert. At every training step, assign the available minibatch slots or accelerator workers to groups with the largest weighted deficits, preventing starvation while avoiding highly bursty update streams that can produce optimizer oscillations.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a fixed global learning-rate estimate in an accelerated optimizer with a curvature envelope that depends on the current estimated optimality gap. Use phase restarts and a descent backtracking test so that the method remains safe when the gap or \(H_1\) estimate is inaccurate. The expected benefit is faster progress on objectives whose curvature is large early in training but decreases substantially near a good solution.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Add a periodic coarse optimization phase to SGD or Adam that operates on a compressed parameterization and returns a prolongated correction to the full network. Retain nonsmooth constraints or regularizers explicitly through a primal-dual update instead of relying on penalty smoothing. Accept the correction only when it improves a cheap fine-batch merit test, making the method useful even when the coarse objective is only approximately coherent with the fine objective.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Train the critic on the action that the environment actually received after safety filtering, not only on the actor's nominal action. Prioritize transitions whose estimation residual, barrier proximity, or filter intervention is large, so replay concentrates on the distribution shift introduced by the safety controller instead of repeatedly sampling benign nominal behavior.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Use the paper’s observation that the fully irreducible vertex is approximately local after crossed-channel ladders are removed to build a block-local curvature correction for neural-network optimization. Estimate a cheap bare covariance and subtract the inverse full covariance to obtain a local irreducible correction, avoiding a dense four-point model while retaining interaction effects that ordinary diagonal preconditioners miss.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace a standard diagonal optimizer preconditioner with a small Riccati-derived feedback controller for a block of neural parameters. The controller explicitly accounts for update-dependent stochasticity, potentially preventing unstable steps in noisy or strongly coupled training dynamics while permitting larger effective learning rates.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace Euclidean projection onto a smooth convex parameter constraint by a local tangent-halfspace linear minimization step. The method is particularly suitable for convex softmax heads or low-dimensional adapter parameters constrained to an ellipsoid, where projections require matrix solves or expensive nonlinear routines but membership and boundary-normal queries are cheap.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace fixed momentum with an online controller that selects the momentum coefficient from an upper bound on the next-step momentum second moment. The controller lowers momentum when minibatch noise dominates and permits higher momentum when the gradient estimate is stable.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace a Euclidean position-plus-rotation recurrent state with an SE(3)-valued latent pose and predict six-dimensional algebra increments rather than directly regressing a rotation matrix or Euler angles. Jointly propagate a pose covariance and penalize Gaussian chance-constraint violations, so the model learns both a nominal trajectory and feedback-like uncertainty contraction.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Inject scale-mixture noise into SGD by sampling the perturbation magnitude from an inverse-gamma distribution rather than using fixed-variance Gaussian noise. The resulting gradient updates have kappa or Student-t tails, allowing rare large exploratory steps while retaining an explicit control parameter for the Gaussian limit and for the existence of noise moments.
Useful6/10
Difficulty3/10
Novelty6/10
Unverified
2026
Replace an explicit gradient step by an implicit correction using the trajectory derivative \(Dg(\theta)g(\theta)=H(\theta)g(\theta)\), where \(g=\nabla f\) and \(H=\nabla^2 f\). The update should strongly damp high-curvature or stiff modes while preserving fourth-order matching of the local linearized dynamics. Start with a self-contained fourth-order L-stable rational prototype, then compare it with the paper's exact two-stage coefficients after recovering those coefficients from the full…
Useful6/10
Difficulty7/10
Novelty7/10
Unverified
2026
Replace ordinary reverse-mode differentiation through a long sequence of stiff relaxation updates with a projected adjoint that separates slow conserved features from rapidly relaxing residual features. The neural layer can use large outer time steps even when its internal relaxation time is very small, while reconstructing only the microscopic gradient component required by the preceding layer.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace a manually tuned penalty optimizer with an inexact augmented-Lagrangian optimizer for neural parameters subject to exact linear constraints such as parameter tying, zero-sum filters, conservation constraints, or structured adapter constraints. Each outer iteration approximately minimizes the augmented Lagrangian using an accelerated proximal-gradient inner loop, and stops when an explicitly computed stationarity certificate reaches a target determined from the current feasibility…
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Add a low-cost performance monitor to an online-adapted neural network and freeze gradient updates after the monitored error has stayed below a target for a dwell interval. The gate prevents continued low-information updates, which otherwise cause parameter drift under weak excitation, noisy observations, or stationary data. Hysteresis allows adaptation to restart after a genuine performance deterioration.
Useful6/10
Difficulty3/10
Novelty6/10
Unverified
2026
Replace communicated floating-point gradients in synchronous federated or data-parallel training with one-bit threshold queries whose thresholds are sampled publicly before gradients are observed. Use several fixed geometric amplitude scales so the same protocol handles unknown gradient means and heavy-tailed client updates without an interactive localization round. Decode each coordinate from the scale whose neighboring estimates are statistically consistent, then apply the decoded aggregate…
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Augment gradient descent with stochastic relocations to uniformly sampled historical parameter vectors. In expectation, the optimizer receives a non-Markovian correction toward the running average of all previous iterates, which can suppress runaway directions and revisit earlier basins instead of remaining trapped in a sharp or unstable region.
Useful6/10
Difficulty4/10
Novelty7/10