✗ Failed on benchmark
2026
Replace a fixed optimizer memory order with a nested family of gradient-integral controllers. Training begins with a first-order update and activates additional accumulated-gradient states only after an exponentially smoothed residual fails to decrease for several decision intervals; newly activated gains are ramped from zero, so the parameter update remains continuous and previously learned states are preserved. The optimizer should use little memory on easy problems and acquire longer memory…
Useful8/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Introduce an effective learning-rate, gain, or regularization parameter that follows the commanded target with a finite implementation rate, and compensate for its predictable threshold-crossing lag. The scheduler estimates the network's current spectral instability boundary and commands the target parameter to cross that boundary early enough that the effective parameter crosses it at the desired time, avoiding overshoot caused by optimizer or hardware smoothing.
Useful8/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace the pointwise strong-form PINN loss with a vector of localized weak residuals generated by fixed compactly supported polynomial test functions. Use a neural network or KAN as the trial function, integrate by parts once, and evaluate each test residual with Gauss–Legendre quadrature; this lowers the required derivative order and prevents a few high-curvature collocation points from dominating training.
Useful8/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Replace a dissipative optimizer update with a canonical discrete flow on the extended state $(\theta,p,t,e)$, where $\theta$ are network parameters, $p$ is momentum, $t$ is training time, and $e$ is its conjugate energy variable. Use a symmetric composition of exact Hamiltonian subflows for kinetic energy, loss, and time translation; this preserves the extended symplectic form and avoids artificial phase-volume collapse. Weak restarts or occasional damping can be added separately if convergence…
Useful8/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Replace a fixed gradient-clipping threshold or fixed optimizer trust region by a dynamic envelope that expands when proposed parameter updates are repeatedly clipped, contracts after clipping disappears, and tightens further during sustained unsaturated convergence. This transfers the paper's bidirectional modification mechanism to training while retaining an explicit safety cap on the actual parameter update.
Useful7/10
Difficulty4/10
Novelty6/10
✗ Failed on benchmark
2026
Replace an instantaneous largest-eigenvalue learning-rate ceiling with a delayed-instability monitor for a slowly ramped optimizer or network gain. When a dominant complex eigenpair crosses from negative to positive real part, permit a controlled post-crossing interval, but stop or roll back when the accumulated positive growth budget exceeds the perturbation/noise margin. This exploits slow-passage delay without allowing unbounded training instability.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Modify an L-BFGS curvature pair only when the observed secant curvature is negative. Replace the gradient-difference vector by the smallest Euclidean or inverse-metric correction that enforces positive curvature, then use the unmodified BFGS update and two-loop recursion. This avoids the computational and conditioning cost of adding a large isotropic damping term to the whole inverse-Hessian approximation.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Use the forward-backward reversal error as an online reliability signal: save more checkpoints or increase the low-rank dimension only when reversing a block produces a large defect. This turns the paper's observations about chaotic low-rank trajectories and rank deficiency into an adaptive memory-versus-gradient-accuracy controller.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace the raw gradient step for a neural-network parameter block with a proximal quasi-Newton step, using the proximal operator to enforce nonsmooth constraints or structured regularization and an adaptive linesearch that enlarges the stepsize after several successful iterations. The method should permit much larger steps than conservative monotone backtracking while retaining a residual-decrease safeguard near unstable regions.
Useful7/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Add a bifurcation-aware monitor or regularizer to a continuous-time recurrent model by evaluating the trace and determinant of its local state Jacobian along the Jacobian kernel direction. Near a nilpotent rank-one equilibrium, these quantities estimate the Bogdanov-Takens coefficients a and b, allowing training to avoid uncontrolled higher-order degeneracies or deliberately target a controlled phase transition in latent dynamics.
Useful7/10
Difficulty5/10
Novelty8/10
✗ Failed on benchmark
2026
Represent training near a switching condition as two locally smooth optimizer modes, such as low- and high-momentum updates or two preconditioners, with a delayed gate. Estimate the leading return-map coefficient and use the paper's scaling law to cap the delay or hysteresis width before an attracting optimization oscillation becomes large. The controller can also intentionally permit a small predicted cycle near saddles or plateaus, then remove the delay as soon as the measured cycle amplitude…
Useful7/10
Difficulty6/10
Novelty8/10
✗ Failed on benchmark
2026
Run multiple optimizer workers, neural-network branches, or expert replicas with delayed parameter messages, using diffusive coupling for agreement and a separately slowed local gradient vector field. The delay should preserve the collective descent direction to first order while multiplying its evolution speed by a predictable factor, allowing communication-delay robustness to be tested independently from ordinary stale-gradient behavior.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace the usual momentum schedule in a neural-network optimizer with a discretization of the paper's lemniscate-acceleration ODE. The method uses a time-dependent friction coefficient that is initially very large and then decays according to lemniscate sine and cosine functions, targeting faster reduction of the gradient norm than constant-momentum SGD or standard Nesterov schedules.
Useful7/10
Difficulty5/10
Novelty8/10
✗ Failed on benchmark
2026
Train an augmented latent neural ODE from snapshot observations of only the visible coordinates by transporting particles from an initial latent distribution and differentiating their visible locations through forward sensitivity equations. Replace density-PDE discretization or potentially biased same-particle density objectives with a kernel marginal-matching loss whose gradient is estimated using independent particle sets.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace pointwise curvature-based learning-rate decisions with a slow-fast entry-exit scheduler. The optimizer maintains a slowly varying state representing effective curvature or gradient-noise level, accumulates the weak transverse growth rate along that slow trajectory, and changes learning regime only when the accumulated rate returns to zero. This permits controlled passage through locally unstable or poorly conditioned regions while preventing indefinite residence in a regime with net…
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Augment an optimizer with a periodic phase and deliberately use a cyclic learning-rate or momentum forcing whose averaged dynamics have an attracting low-dimensional set. Treat the resulting periodic parameter orbit as an invariant torus and tune the schedule so transverse contraction dominates tangential sensitivity and minibatch perturbations. The goal is a robust, phase-locked training orbit that explores parameter space without losing attraction toward a useful solution manifold.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace the unresolved small jumps of an infinite-activity stable Levy noise source in a neural SDE or stochastic optimizer with one Gaussian increment whose variance equals the discarded jump variance. Simulate only jumps above the cutoff exactly or by Poisson sampling, retaining the large-jump distribution while obtaining the paper's O(\varepsilon) Wasserstein error instead of the naive O(\varepsilon^{1-\alpha/2}) error.
Useful7/10
Difficulty5/10
Novelty8/10
✗ Failed on benchmark
2026
Use the paper's sharp sK approximately equal to 1 phase transition to choose between conservative Fejer averaging and higher-order polynomial filtering. When the local fixed-point spectrum is separated from eigenvalue 1, use a Jackson-type filter; near the critical regime, use the safe Fejer filter instead of unrestricted Anderson extrapolation.
Useful7/10
Difficulty7/10
Novelty7/10
✗ Failed on benchmark
2026
Implement a momentum optimizer as a contact Hamiltonian splitting rather than as a direct Euler discretization. Introduce an auxiliary scalar contact state and compose exact kinetic, potential, and damping subflows; this produces a second-order conformal integrator whose modified contact energy should decay more reliably at moderately large learning rates.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace independently restarted proximal-gradient or quasi-Newton solves for a composite neural objective with a curvature-recycling Douglas–Rachford loop. The previous proximal state, residual, and limited-memory BFGS curvature pairs are transported to the next proximal center, reducing expensive loss and gradient evaluations while retaining the cheap nonsmooth proximal operation.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace a neural-network optimizer's globally fixed learning-rate geometry with an adaptive quadratic trust region. At every update, construct a local curvature model, accept or reject the step using the ratio between realized and predicted loss decrease, and expand or contract the radius accordingly; the same controller should automatically become conservative in nonconvex regions and Newton-like near a well-conditioned minimum.
Useful7/10
Difficulty6/10
Novelty6/10
✗ Failed on benchmark
2026
Use the paper's finding that smooth approximations of a circular current limiter can generate spurious Hopf bifurcations to audit smooth bounded operations in optimizers and networks. Compare exact projection with a differentiable surrogate, continue both dynamics in clipping threshold or step size, and reject a surrogate if it introduces a unit-circle crossing absent from the exact map.
Useful7/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Replace a fixed distributed gradient-reduction topology with a topology selected from recent estimates of each shard's mean and variance. The selector minimizes the paper's predicted second-moment rounding cost, favoring shallow placement for high-variance coordinates and Huffman-like placement for unequal-variance blocks, while retaining the same number of additions and communication volume.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Wrap a recurrent or state-space neural network in a sampled-data feedback loop: latent states evolve continuously or at every fine solver step, while a constrained optimizer updates the control, adapter, or residual-gating vector only every M steps. Between optimizer updates, use zero-order hold or linear interpolation and reject updates that violate a learned Lyapunov decrease condition. This should prevent large transient latent explosions caused by aggressive optimizer updates while…
Useful7/10
Difficulty6/10
Novelty7/10