✗ Failed on benchmark
2026
Replace the raw gradient step for a neural-network parameter block with a proximal quasi-Newton step, using the proximal operator to enforce nonsmooth constraints or structured regularization and an adaptive linesearch that enlarges the stepsize after several successful iterations. The method should permit much larger steps than conservative monotone backtracking while retaining a residual-decrease safeguard near unstable regions.
Useful7/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Replace explicit zero-padding before FFT convolution by the paper's mixed-radix decomposition, which injects zeros through bounded tile sums and never allocates the padded input. The resulting transform is mathematically identical to the length-M transform of the explicitly padded signal, while reducing temporary storage and potentially memory bandwidth.
Useful7/10
Difficulty7/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Introduce a periodic modulation of the local linearized training or inference dynamics and choose its frequency and amplitude using spectral stability measurements. In the slow regime, stability should be predicted by the time average of the instantaneous rightmost eigenvalue; in the fast regime, periodic modulation may suppress growth through a noncommuting, high-frequency Floquet correction even when individual instantaneous Jacobians are unstable.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Use the paper's structure-exploiting primal-dual active-set strategy to solve barrier-constrained neural updates without invoking a generic quadratic-program solver at every step. The active constraints identify which layers or state statistics are actually close to instability, while warm-started multipliers and active sets should make the safety correction nearly constant-cost when the training trajectory changes smoothly.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Insert a linear Johnson–Lindenstrauss bottleneck around a set of jointly processed representations, choosing its width from the sharp finite-set dimension bound rather than from the model's nominal hidden size. The projection should preserve pairwise distances between tokens, patches, or retrieved items, allowing a downstream attention or MLP block to operate at lower width while retaining the geometry relevant to similarity computations.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Use trajectory sensitivities to remove neural units or parameter groups whose effects are redundant over the available data support. A parameter group is pruned when its Fisher contribution is small or its sensitivity is nearly collinear with other groups, producing a compact neural ODE without relying only on parameter magnitude.
Useful7/10
Difficulty5/10
Novelty8/10
✗ Mechanism failed
2026
Replace dense cross-attention weights with a balanced transport plan whose nonzero query-key edges are maintained by a multiscale active-set procedure. Solve the coarse token-group problem first, lift its support to the fine token grid, add only edges indicated by local cost or marginal residuals, and warm-start the fine problem from the lifted plan. This should provide a principled sparse attention pattern rather than fixing a global top-k pattern before seeing the transport solution.
Useful7/10
Difficulty7/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace multiplicative query-key attention scores with an affinity based on the l1 distance between first-spike latency vectors. For each query token and key token, small latency differences produce large affinity and distant timings decay exponentially, yielding a locality-sensitive attention pattern naturally compatible with leaky spiking neurons.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Build a neural operator around explicit input and output measurement spaces rather than forcing the network to consume and emit a fixed grid. The same learned latent surrogate can be reused on alternative sensor layouts or query meshes through reconstruction and re-encoding maps, with a consistency loss enforcing agreement between measurement pipelines.
Useful7/10
Difficulty6/10
Novelty5/10
✓✓ Beats tuned baseline
2026
Attach a polyharmonic spline decoder to a coordinate MLP or use it as a standalone neural-field output head over a large set of spatial anchors. The decoder represents the output as a low-degree polynomial trend plus a PHS kernel expansion, while FMM evaluates all anchor-to-query interactions in approximately linear or near-linear cost. When coefficients must be fitted or periodically recalibrated, solve the constrained interpolation system with projected conjugate gradients and a sparse…
Useful7/10
Difficulty6/10
Novelty7/10
✗ Mechanism failed
2026
Use the interpolation SDP to synthesize coefficients for a short-memory first-order minimax optimizer with a certified worst-case contraction rate. The resulting recurrence can combine current and previous iterates and gradients, providing an offline-designed alternative to hand-tuned simultaneous descent-ascent, extragradient, or optimistic-gradient updates.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Mechanism failed
2026
Evaluate weak residuals against a bank of periodic trigonometric test functions using FFT projections instead of repeated pointwise quadrature or output automatic differentiation. Frequency truncation and mode weighting provide a direct way to control the spatial scales enforced during neural PDE training.
Useful7/10
Difficulty4/10
Novelty6/10
✗ Failed on benchmark
2026
Replace a diagonal learning-rate or preconditioner matrix with a small full block matrix and communicate a worker's updated gradient or parameter only when its local state has drifted sufficiently from the last communicated state. Jointly select the block preconditioner and the largest safe trigger threshold using robust Lyapunov inequalities over several empirical Hessian or Gauss-Newton matrices. The expected gain is fewer synchronization events without the instability normally caused by…
Useful7/10
Difficulty7/10
Novelty7/10
✗ Mechanism failed
2026
Train a small encoder and latent Koopman predictor to forecast whether a neural sequence model will enter a high-error or high-instability region, then execute an expensive refinement block only when the forecasted risk exceeds a threshold. The base model remains active at every step, so the learned preview model controls computation rather than directly replacing the main predictor. Add a bounded-rate interpolation when the gate switches off, preventing abrupt changes in recurrent state or…
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Run multiple optimizer workers, neural-network branches, or expert replicas with delayed parameter messages, using diffusive coupling for agreement and a separately slowed local gradient vector field. The delay should preserve the collective descent direction to first order while multiplying its evolution speed by a predictable factor, allowing communication-delay robustness to be tested independently from ordinary stale-gradient behavior.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Construct a learned optimizer whose update is an ordered sequence of local implicit parameter-block solves, then differentiate the finite optimization trajectory with reverse local adjoints. This enables training optimizer hyperparameters or meta-gradients through many inner steps without storing all intermediate tensor operations or replacing the executed trajectory by an idealized fixed-point gradient.
Useful7/10
Difficulty6/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Choose an initialization that may have worse initial loss but has a smaller projection onto the slow modes of the subsequent training dynamics. Under the same optimizer, data order, and learning rate, this initialization should overtake a lower-loss baseline after a predictable crossing time, analogous to the paper's reversal of relaxation ordering.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Use the lifted second-moment operator to adapt the residual step size of a deep residual network or neural ODE under multiplicative layer noise. Instead of choosing a fixed residual coefficient, shrink or enlarge it online to keep the predicted covariance-growth factor below a target margin, producing a stochastic stability controller for depth and inference time.
Useful7/10
Difficulty5/10
Novelty8/10
✗ Failed on benchmark
2026
Replace hand-designed Heavy Ball or Nesterov coefficients with a low-order linear controller synthesized by a semidefinite program. The controller receives the stochastic mini-batch gradient and emits the parameter update; dynamic IQC multipliers constrain both gradient curvature and temporally correlated mini-batch noise, so the SDP directly minimizes a certified contraction factor rather than optimizing momentum heuristically.
Useful7/10
Difficulty7/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Compress only the difference between the current client gradient and a persistent control variate, rather than compressing the full gradient. As the control variate tracks the client gradient, the residual shrinks and the same communication budget produces less compression noise than direct gradient quantization.
Useful7/10
Difficulty4/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Use the negative polar factor of a tangent-projected momentum matrix as the update direction for a weight matrix constrained to the Stiefel manifold. After taking the step, apply a polar retraction so the columns remain exactly orthonormal, avoiding penalty losses and constraint drift.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace fixed-length binary dot products with accumulations whose terms are processed in descending order of weight magnitude. Stop as soon as the current partial sum is larger in magnitude than the total absolute magnitude of all remaining terms; the output sign is then guaranteed to equal the full dot-product sign, eliminating unnecessary additions without changing accuracy.
Useful7/10
Difficulty4/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace full agent-to-agent state transmission with coefficients in a learned dominant Koopman mode basis. Agents communicate only the leading spectral coordinates that explain slowly decaying collective behavior, while retaining a certificate based on the spectral gap and subdominant eigenvalue to decide whether the compressed representation is safe.
Useful7/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Add an explicit gradient feedthrough D to a momentum optimizer and choose it below the estimated inverse smoothness, D < 1/L. Use the resulting passivity margin to govern momentum: increase the momentum-channel gain only while the measured storage dissipation remains nonnegative, and reduce the feedthrough or momentum when the passivity residual becomes positive.
Useful7/10
Difficulty5/10
Novelty7/10