✗ Failed on benchmark
2026
Replace a fixed optimizer memory order with a nested family of gradient-integral controllers. Training begins with a first-order update and activates additional accumulated-gradient states only after an exponentially smoothed residual fails to decrease for several decision intervals; newly activated gains are ramped from zero, so the parameter update remains continuous and previously learned states are preserved. The optimizer should use little memory on easy problems and acquire longer memory…
Useful8/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Add a controllable delay to the gradient force during optimization so that parameters follow a delayed-gradient dynamical system. Choose the delay below the stability boundary for ordinary training, and temporarily cross the boundary when the optimizer is trapped in a sharp or stagnant basin, causing stochastic fluctuations to be amplified out of the basin rather than waiting for a rare Arrhenius escape.
Useful8/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Introduce an effective learning-rate, gain, or regularization parameter that follows the commanded target with a finite implementation rate, and compensate for its predictable threshold-crossing lag. The scheduler estimates the network's current spectral instability boundary and commands the target parameter to cross that boundary early enough that the effective parameter crosses it at the desired time, avoiding overshoot caused by optimizer or hardware smoothing.
Useful8/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Use the generalized Cramér–Rao relation to adapt the inverse-temperature or noise schedule of an energy-based sampler, diffusion sampler, or stochastic optimizer. The controller limits each temperature change according to the measured energy variance and Fisher information, preventing uncontrolled changes in the sampled energy distribution while allowing larger steps in insensitive regions.
Useful8/10
Difficulty4/10
Novelty7/10
✗ Failed on benchmark
2026
Model one period of a cyclic optimizer or periodically modulated recurrent network as a discrete-time linear time-periodic system obtained by linearizing the update around its current trajectory. Estimate a periodic Lyapunov matrix sequence and scale the next learning-rate or modulation amplitude so that every phase contracts according to a certified energy decrease. This should prevent delayed divergence caused by resonance with the schedule, even when individual phase Jacobians are…
Useful8/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace a fixed learning rate for each layer or parameter block with a bounded gain selected by the one-step-ahead predictive loss. The sign of the product between the current gradient and the next gradient estimates whether the previous update moved downhill: aligned gradients increase the gain, while sign reversals decrease it. A mirror-descent update on a bounded interval prevents the runaway step sizes that can occur with exponential or unconstrained learning-rate parameterizations.
Useful8/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Learn a low-dimensional Koopman operator from successive states of an iterative neural system, such as debate agents, recurrent refinement blocks, or diffusion denoising trajectories. Use the magnitude of the subdominant eigenvalue to predict the remaining number of rounds required for disagreement to fall below a target tolerance, and stop computation when the predicted deadline is reached rather than using a fixed round budget.
Useful8/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace an instantaneous diagonal optimizer with a causal convolution of recent gradients, where cross-layer or cross-module gradient correlations define a finite-memory Onsager response matrix. Estimate the response at several parameter-block pairs and lags, integrate it to obtain a finite-time transport matrix, and use its regularized inverse or symmetric part to precondition the update. This targets optimization regimes in which gradients propagate between blocks with measurable delay, such…
Useful8/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace periodic all-reduce in federated or distributed training with local broadcasts triggered by a prescribed parameter-disagreement envelope. Each worker maintains held copies of the latest parameters received from neighbors and applies a consensus correction to its local optimizer update. After an asynchronous reception causes a discontinuous change in sampled disagreement, a receiver-side exponentially decaying correction temporarily enlarges the allowable envelope, preventing false…
Useful8/10
Difficulty6/10
Novelty8/10
✗ Failed on benchmark
2026
Replace a slow sequence of resolvent or contractive fixed-point updates by a blockwise averaged-reflection extrapolation. The method computes reflected iterates R^j y_0, averages them with equal weights, and uses the result as the next macro-iterate. Unlike unconstrained Anderson acceleration, this construction has a uniform residual guarantee for every maximal monotone operator.
Useful8/10
Difficulty4/10
Novelty6/10
✗ Failed on benchmark
2026
Replace each recurrent neural state with two asymmetrically coupled variables: a slow state x_i and a fast momentum or drive variable v_i. Each coordinate or block updates independently using its locally available, possibly stale input; the auxiliary variable supplies inertia that suppresses harmful update-order sensitivity and can accelerate traversal toward a retrieved state or denoised solution.
Useful8/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Model a multi-timescale optimizer as a controlled dynamical system and use several Lyapunov-like quantities to regulate loss, momentum energy, and constraint violation simultaneously. The explicit high-order control-Lyapunov feedback becomes a low-cost correction to an SGD-momentum or Adam step. A Hurwitz comparison matrix supplies a measurable stability certificate and predicts the decay rate of the controlled training dynamics.
Useful8/10
Difficulty6/10
Novelty8/10
✗ Failed on benchmark
2026
Replace an EKF or a large particle ensemble inside a neural world model with a fixed-order polynomial chaos representation of the latent state distribution. The transition network is evaluated under quadrature or sampled chaos variables, and Galerkin projection produces the next uncertainty coefficients directly; a coefficient-wise LMMSE update then assimilates observations without backpropagating through resampling.
Useful8/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace recurrence or nearest-neighbour analogue lookup with a learned delay-coordinate observer that continuously corrects a latent state using the current observation. Constrain the observer's closed-loop Jacobian or linear state matrix to have spectral radius below one, so prediction error contracts geometrically and required burn-in grows logarithmically with target accuracy.
Useful8/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
For a recurrent or graph neural network with known local connectivity, estimate each node's local Jacobian row using only graph neighbors rather than all hidden coordinates. Use the resulting sparse Jacobian both to compute a contraction certificate and to regularize training toward dynamically local interactions, reducing estimator variance and the number of samples required for reliable stability decisions.
Useful8/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace ordinary global gradient clipping with clipping of each stochastic gradient around a robust minibatch center rather than around zero. This preserves the common directional component of the gradients and suppresses only heavy-tailed residuals, making the update usable when gradient noise has a finite α-moment for 1 < α ≤ 2 but no finite variance.
Useful7/10
Difficulty4/10
Novelty6/10
✗ Failed on benchmark
2026
Turn constrained-flow generation efficiency into an online diagnostic and controller for neural sampling. When the target ensemble changes faster than the flow can track or becomes internally complex, automatically shorten the training window, increase flow updates, or fall back to local MCMC instead of silently accepting biased or highly correlated samples.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Add a fading-memory consensus force to parameter dynamics, pulling the current parameter toward a distributed average of its past while preserving the ordinary gradient step. Implement the infinite memory through one or several recursive exponential states, and tune the memory decay so that quadratic-mode dynamics remain exponentially stable. This should suppress oscillations and catastrophic steps without relying on conventional momentum alone.
Useful7/10
Difficulty4/10
Novelty6/10
✗ Failed on benchmark
2026
Replace an instantaneous largest-eigenvalue learning-rate ceiling with a delayed-instability monitor for a slowly ramped optimizer or network gain. When a dominant complex eigenpair crosses from negative to positive real part, permit a controlled post-crossing interval, but stop or roll back when the accumulated positive growth budget exceeds the perturbation/noise margin. This exploits slow-passage delay without allowing unbounded training instability.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Use the paper's extreme-value escape statistics as a diagnostic for delayed-gradient bursts. If many stochastic minibatch realizations escape through an unstable delay mode, their first-passage times should become approximately Gumbel distributed, allowing the optimizer to distinguish useful basin escape from destructive divergence and to terminate or retune the burst automatically.
Useful7/10
Difficulty5/10
Novelty8/10
✗ Failed on benchmark
2026
Modify an L-BFGS curvature pair only when the observed secant curvature is negative. Replace the gradient-difference vector by the smallest Euclidean or inverse-metric correction that enforces positive curvature, then use the unmodified BFGS update and two-loop recursion. This avoids the computational and conditioning cost of adding a large isotropic damping term to the whole inverse-Hessian approximation.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace a full Hermitian curvature matrix, such as a Hessian or empirical Fisher matrix, by its block-diagonal version only when the paper's perturbation certificate predicts a small eigenvalue change. Use the certificate online to merge poorly separated blocks and retain independent preconditioners for well-separated blocks, yielding a controllable accuracy-memory tradeoff rather than a fixed block-diagonal approximation.
Useful7/10
Difficulty6/10
Novelty5/10
✗ Failed on benchmark
2026
Wrap stochastic optimization or iterative neural inference in a controller that measures how far the state distribution moves during each interval and compares this motion with the available noise-dependent entropy-production budget. The controller increases the learning rate or reduces inference steps only while the trajectory remains inside the predicted speed-limit region, preventing fast jumps that cause accuracy collapse.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Partition neural-network parameters into competing blocks, such as LoRA adapters, mixture-of-experts heads, or task-specific heads, and update each block by minimizing its local quadratic model while holding the other blocks fixed. Use the exact Jacobi coupling spectral radius to decide whether simultaneous updates are stable; near the boundary, apply damping or fall back to sequential Gauss-Seidel updates.
Useful7/10
Difficulty5/10
Novelty7/10