✓✓ Beats tuned baseline
2026
Replace every-step parameter communication or correction by an impulsive update emitted only when the local optimization state has drifted sufficiently from its last transmitted value. The correction is executed after a known or measured delay, and the trigger threshold is selected so that stale updates remain inside a Lyapunov-certified stability region while reducing communication and redundant optimizer work.
Useful8/10
Difficulty5/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Treat optimization as a forced dynamical system whose state is the parameter velocity and whose input is the minibatch gradient. Permit ordinary momentum updates below a target energy, but smoothly increase damping when optimizer energy exceeds that target. This preserves less-conservative behavior in low-energy regions while imposing dissipative dynamics during potentially divergent excursions.
Useful8/10
Difficulty4/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Replace the standard unit-step modern Hopfield retrieval update with a relaxed step using theta greater than 1, while restricting theta to the theoretically safe interval (0,2). The relaxed map has the same fixed points as ordinary attention and provably decreases the Hopfield energy, so it can move farther toward an attractor per iteration without changing the retrieval objective.
Useful8/10
Difficulty3/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Partition graph nodes into backward-equivalent classes and run message passing on the K-node quotient graph instead of the original N-node graph. If every node in a class receives the same aggregate message from every source class and shares the same local update map, class-constant node representations remain class-constant at every layer, making the quotient computation exactly equivalent to the full GNN on that invariant subspace.
Useful8/10
Difficulty5/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Train two parameter replicas with symmetric coupling, treating one replica as a prepared thermalization packet for the other. Estimate the slow local Hessian direction and initialize or periodically reset the packet so that the coupled state has zero projection onto that mode; the target should then relax according to the next-slowest mode rather than the original bottleneck.
Useful7/10
Difficulty6/10
Novelty8/10
✓✓ Beats tuned baseline
2026
Introduce a periodic modulation of the local linearized training or inference dynamics and choose its frequency and amplitude using spectral stability measurements. In the slow regime, stability should be predicted by the time average of the instantaneous rightmost eigenvalue; in the fast regime, periodic modulation may suppress growth through a noncommuting, high-frequency Floquet correction even when individual instantaneous Jacobians are unstable.
Useful7/10
Difficulty6/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Monitor the ratio between gradient norm and square-root loss suboptimality, and use it to distinguish the far-from-optimum linear-decay regime from the near-optimum exponential regime predicted by semiglobal PŁI. Apply conservative updates or gradient clipping while the ratio is small, then switch to a larger stable learning rate, reduced gradient noise, or early stopping once the local PŁI regime is detected.
Useful7/10
Difficulty4/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Replace a single global learning rate with mode-dependent rates determined by the static correlation structure of recent parameter updates or hidden-state updates. Correlated modes are treated as collective diffusive modes: their effective relaxation rate is reduced in proportion to their structure-factor amplitude, so the optimizer accelerates weakly correlated modes while damping collective slow modes. The method also supplies a diagnostic for when the Markovian approximation is invalid and…
Useful7/10
Difficulty6/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Use bounded-noise escape as a measurable stability transition to adapt the learning rate or recurrent integration step before catastrophic loss of confinement. Periodically estimate the disturbance radius at which the current training dynamics exits its stable region, then adjust the step size to maintain a fixed safety margin.
Useful7/10
Difficulty6/10
Novelty9/10
✓✓ Beats tuned baseline
2026
Replace constant decoupled weight decay with a coefficient proportional to the current learning rate divided by the peak learning rate. The optimizer applies ordinary decay at the learning-rate peak but weakens decay during cooldown and late training, preventing unnecessary steady-state parameter-norm shrinkage while retaining early-training stabilization.
Useful7/10
Difficulty2/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Replace a fixed SGD learning rate with a per-update step selected from the positive curvature observed along the proposed direction. The controller estimates the directional Taylor remainder using one or two function evaluations, increases the step when the observed direction is benign, and backtracks only when the update fails a sufficient-decrease test.
Useful7/10
Difficulty5/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Use the paper's lifetime law as a controller for training or rollout difficulty. Estimate the active perturbation bandwidth R of hidden states or forecast errors and reduce the residual gain, increase the dispersion order W, or inject controlled bandwidth whenever the estimated prethermal lifetime becomes too short.
Useful7/10
Difficulty5/10
Novelty8/10
✓✓ Beats tuned baseline
2026
Replace an unconstrained recurrent transition with a hierarchy of features whose generator is triangular: degree-ell features depend only on degree-ell and lower-degree features. This transfers the paper's closure mechanism for even-Majorana monomials into a neural state-space model, preserving nonlinear feature interactions while making the spectrum and long-time transients directly controllable.
Useful7/10
Difficulty5/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Replace ordinary randomized coordinate descent inside a least-squares neural subproblem with RPLSS's projected direction update. Each sampled parameter coordinate generates a Jacobian column, while the stored matrix P removes components already covered by previous updates; this should reduce redundant coordinate steps and improve convergence for linear heads, LoRA modules, and locally linearized fine-tuning.
Useful7/10
Difficulty6/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Use the differentiable covariance chart to construct a Fisher-information preconditioner for the edge and innovation parameters of a linear-Gaussian neural module. Instead of applying an isotropic Euclidean update, whiten parameter steps according to how strongly they change the predicted Gaussian distribution. This targets ill-conditioning caused by redundant paths, correlated latent nodes, and badly scaled innovation covariances.
Useful7/10
Difficulty6/10
Novelty5/10
✓✓ Beats tuned baseline
2026
Separate a neural network into nonlinear hidden parameters and a linear output layer. Solve the output layer exactly by least squares, then update hidden parameters with a truncated-pseudoinverse Gauss-Newton step that discards numerically singular directions.
Useful6/10
Difficulty6/10
Novelty5/10