✗ Failed on benchmark
2026
Replace Gaussian perturbations in a low-dimensional neural-network optimizer with independent double-geometric integer mutations and adapt each mutation scale using its exponential-family natural gradient. Apply the method to layerwise quantization scales, adapter coefficients, pruning thresholds, or other integer/discrete hyperparameters rather than to every individual weight.
Useful6/10
Difficulty5/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Add a geometry-guided infill operator to a population optimizer used for black-box neural-network tuning. Fit a local Jacobian from recent parameter perturbations and validation-residual vectors, generate a damped Gauss-Newton candidate for exploitation, and sample exploratory candidates in the same Jacobian-derived metric. The host optimizer retains selection, population survival, covariance adaptation, and its total evaluation budget; only a configurable fraction of new candidates is replaced…
Useful6/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Replace AdamW or SGD updates on simplex-valued routing probabilities with a logarithmic-barrier mirror step. The update remains strictly positive, avoids projection-induced zero coordinates, and can approach a boundary solution asymptotically while retaining the paper's theoretically motivated O(log k/k) convex convergence behavior.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Use state-dependent Langevin noise to perturb prototype or router-codebook optimization when balanced quantization reaches a poor stationary configuration. Retain the best-so-far state for deployment, allowing exploration to escape non-minimizing Lloyd fixed points without permanently corrupting the learned representation.
Useful5/10
Difficulty4/10
Novelty5/10
Unverified
2026
Use a Goldfarb–Idnani-style active-set solver as a neural constrained layer or optimizer substep, but never trust a guessed active set solely because its linear system solved. Remove duplicate or dependent constraints, solve the reduced KKT system, and accept the result only after checking primal feasibility, dual sign conditions, and stationarity. This gives exact enforcement of linear inequalities and a diagnostic certificate when the constraint set is infeasible.
Useful5/10
Difficulty6/10
Novelty5/10
Unverified
2026
Replace a hard nonnegative slack or ReLU output by the barrier-derived map \(x_s(w)=\frac{w+\sqrt{w^2+4s}}{2}\). Unlike an arbitrary smooth activation, this output is the unique positive solution of \(x(x-w)=s\), so the network can explicitly monitor complementarity and anneal \(s\) toward the true inequality-constrained solution. Use it in a constrained output head or in hidden layers whose activations represent nonnegative resource, probability, or routing slack variables.
Useful5/10
Difficulty3/10
Novelty4/10
Unverified
2026
Use the Gaussian approximation of a high-dimensional maximum to set a simultaneous coordinate-clipping threshold for minibatch gradients or activations. The threshold is sampled from a correlated Gaussian with the observed batch covariance, rather than treating coordinates as independent or estimating an unstable extreme quantile directly.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Treat the number K of minibatches between expensive control updates as a review period: the controlled neural dynamics use parameters or decisions computed at time nK and hold them fixed until (n+1)K. Scan K, estimate first and second finite differences of validation loss or episodic return, and use the resulting nonmonotone-to-convex or concave phase diagram to select an update frequency rather than assuming that more frequent updates are always better.
Useful5/10
Difficulty4/10
Novelty6/10
Unverified
2026
Train a neural approximation to a scale-dependent effective action, energy functional, or field while penalizing the residual of a known continuous-symmetry Ward identity. Select the regulator, smoothing scale, or architecture hyperparameter at the minimum Ward residual, and require that the residual decreases when model capacity or derivative-expansion order increases.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Use a pressure objective to select expert-routing distributions by balancing task reward against route entropy, rather than optimizing task loss alone. The resulting router behaves like an equilibrium-state estimator: it should retain multiple high-performing branches when their combined entropy outweighs the advantage of a single branch.
Useful5/10
Difficulty5/10
Novelty5/10
Unverified
2026
Use spatially correlated training points whose low-frequency structure factor vanishes instead of iid points. For neural fields, PINNs, image-coordinate MLPs, or spatially indexed minibatches, this should suppress long-wavelength quadrature and gradient-estimation noise while preserving the represented target dynamics. The finite-order prediction is that a design with structure factor S(k)=O(|k|^{2q}) produces lower variance for smooth losses than iid sampling, especially as the domain or batch…
Useful5/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace raw updates of strongly coupled parameter blocks by updates in rescaled, approximately normal-form coordinates. The optimizer estimates the local coupling matrix between block directions, solves a small modulation system for transformed velocities, and optionally subtracts predictable first-order cross-block drift.
Useful5/10
Difficulty5/10
Novelty4/10
Unverified
2026
Parameterize the time-dependent coefficients of a latent neural ODE in a Chebyshev system instead of an unconstrained neural network, and train the resulting Poincare residual to have a prescribed number of simple zeros. If the relevant Melnikov function belongs to a certified Chebyshev span, the model obtains an explicit upper bound on the number of isolated periodic latent trajectories and limits uncontrolled oscillatory behavior.
Useful5/10
Difficulty7/10
Novelty9/10
Unverified
2026
Use multiplier bootstrap on minibatch activation covariances to determine whether a large top eigenvalue is a genuine representation direction or merely a high-dimensional bulk fluctuation. When a spike is repeatedly significant, apply a low-rank whitening or shrinkage correction to that activation subspace; otherwise leave the layer unchanged, avoiding destructive whitening of ordinary bulk variation.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a Euclidean feature-space metric by a short-time heat-kernel/Wasserstein metric and use it to precondition updates or penalize distortions of local neighborhoods. The first-order correction is a Ricci-curvature term, while the second-order residual captures curvature variation and quadratic curvature effects that ordinary diffusion smoothing misses.
Useful5/10
Difficulty7/10
Novelty7/10
Unverified
2026
Add higher-order filtered-error states to parameter-efficient fine-tuning and constrain the highest-order state to a prescribed shrinking funnel. The resulting recursion gives an explicit bound on parameter drift and its filtered derivatives at every lower order, providing a principled alternative to a fixed quadratic proximity penalty or unconstrained momentum.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Augment gradient descent with a directional-search step when the gradient norm is small or the loss has stalled. In each parameter block, evaluate a small positively spanning set of normalized perturbations, use their directional loss slopes to identify descent directions, and combine them through nonnegative coefficients so that the update remains inside their positive span. The cosine measure supplies a quantitative trigger: low directional coverage means the current perturbation pool is not…
Useful5/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace soft pairwise repulsion between learned prototypes or codebook vectors with an active-set feasibility layer based on the paper's first-order admissible cone. Pairs exactly at the minimum distance contribute linear half-space constraints to the update, while separated pairs do not unnecessarily restrict motion. This should reduce prototype collapse and make constrained embedding or quantization training less sensitive to penalty weights.
Useful5/10
Difficulty5/10
Novelty4/10
Unverified
2026
Replace the explicit parameter update \(\theta_{k+1}=\theta_k-\eta\nabla L(\theta_k)\) with an approximate generalized proximal step defined by a simple map \(v\). The map is chosen so that the gradient operator and v satisfy an empirical pair-monotonicity condition, allowing larger stable outer steps and reducing oscillations in stiff or highly curved neural-network training.
Useful5/10
Difficulty7/10
Novelty6/10
Unverified
2026
Replace the ordinary minibatch mean gradient by a coordinatewise quantile-winsorized mean. Each parameter-gradient coordinate is clipped to empirical lower and upper quantiles before aggregation, limiting the influence of adversarial examples while retaining all samples and avoiding the discontinuity of hard trimming.
Useful5/10
Difficulty6/10
Novelty5/10
Unverified
2026
Use differential evolution over adapter or prompt parameters, combining attraction to the current best parameter vector with a population-difference direction. Binomial crossover supplies coordinate-level exploration, while the operator-selection separation makes it possible to measure raw proposal geometry independently from parameter repair and noisy fitness selection.
Useful5/10
Difficulty5/10
Novelty5/10
Unverified
2026
Train Fourier or state-space neural models by eliminating well-conditioned spectral modes first and retaining near-resonant modes until a later stage. The schedule is determined by the small-divisor geometry of a reference transport vector, with a cumulative Brjuno-like budget controlling how aggressively spectral corrections may be applied. This should prevent rare nearly resonant modes from producing disproportionately large gradients or unstable long-horizon rollouts.
Useful5/10
Difficulty5/10
Novelty8/10
Unverified
2026
Replace penalty-based equality-constrained training with a two-timescale optimizer. A fast variable tracks the normal correction that drives constraint residuals toward zero, while the slow parameter update follows the task gradient projected onto the local constraint tangent space. This should reduce sensitivity to very large penalty weights and preserve feasibility more accurately during training.
Useful5/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace ordinary projected-gradient updates for a convex neural subproblem with a homogeneous perspective formulation and Douglas-Rachford splitting. The additional scale variable makes the update less sensitive to large variations in loss or parameter scale and can expose infeasible combinations of constraints instead of producing unstable iterates.
Useful5/10
Difficulty6/10
Novelty7/10