Unverified
2026
Treat a two-player neural game as a stochastic monotone operator problem and add an anchor regularizer that makes the current game strongly monotone during early training. Use stochastic extragradient steps inside geometrically decreasing regularization stages, retaining the accumulated anchor rather than resetting it; this should suppress rotational GAN dynamics while avoiding a permanent bias toward the initialization.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace dense token attention with K asynchronous local nonlinear consensus updates on a sparse token graph. Each selected token is moved to the unique incident p-mean of its neighbors, producing a tunable message-passing operator that is averaging for p=2 and nonlinear for other p.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Propagate uncertain inputs or parameter perturbations through a neural network with zonotopes, and choose the reduction rule after each layer using short lookahead that predicts downstream certified-bound error. Candidate reducers can include largest-generator retention, box conversion, and norm-based merging; unlike a fixed policy, the chosen rule depends on the current generator geometry and remaining network depth. The goal is tighter robustness bounds at the same generator budget, or the…
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Construct a finite-horizon momentum schedule and its coefficient-reversed counterpart, then use the reversed schedule when the desired endpoint criterion is gradient norm rather than function-value decrease. Optimize the schedule on extremal-curvature quadratic proxies, where the dynamics reduce to an exactly simulable scalar recurrence; this creates a cheap, falsifiable optimizer-design procedure without changing the neural-network architecture.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Represent a small ensemble of parameter vectors or a low-dimensional projection of parameters as a periodically driven probability distribution, and optimize the learning-rate/control waveform so that the distribution translates through successive optimization phases at approximately constant speed without unnecessary reshaping. The method adds a dissipation penalty based on probability-current mismatch and uses automatic differentiation through the particle or density dynamics to learn a…
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Replace dense aggregation over patches, points, or graph neighbors with a state-dependent positive quadrature rule. A latent encoder produces the current manifold coordinate z, a small controller chooses a subset of entities, and a convex redistribution step assigns nonnegative weights so that the subset preserves the full aggregation on a learned feature basis. The method is especially suitable for neural operators and vision attention where many spatial entities are redundant along the data…
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace an unconstrained third-order stationarity update by a Chebyshev root step on a one-dimensional restriction of the neural loss, while monitoring the images of the update map's free critical points. Shrink the trust radius or damping parameter whenever the maximum critical-value gain exceeds one, because the paper's mechanism predicts that critical-orbit expansion marks loss of a safe attracting basin.
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Convert the paper's second-order forward-backward dynamics into a two-lag optimizer for neural-network parameters. The update combines an anchor-controlled Halpern step with two inertial differences, allowing the optimizer to use both recent velocity and an older velocity while retaining a tunable contraction toward a reference point.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace unconstrained token-to-expert routing with a nonnegative mixture of a small set of feasible routing or communication patterns. Enforce resource limits using the paper's one-sided positive-contribution bound, which gives a conservative certificate without enumerating all joint token activation scenarios. Expand the pattern set only when a separation procedure finds a routing pattern that improves the router objective while adding useful capacity information.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace the noisy Hutchinson estimate of a neural-network Hessian trace with a variance-reduced Hutch++ estimate computed only from Hessian-vector products. Use the estimated normalized curvature to cap or rescale the optimizer step, so learning-rate reductions occur when the loss landscape becomes globally sharp rather than when an individual minibatch gradient happens to be large.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace the momentum update in a gradient optimizer by inertial motion plus a gradient-difference term, which discretely approximates Hessian-driven damping. Choose the damping coefficient and step size using the paper's refined stability inequality instead of the older restrictive bound, and adapt them whenever the estimated smoothness changes.
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Use two bounded self-adjoint transition operators in a recurrent or state-space block and penalize their normalized Hilbert--Schmidt commutator. When the penalty is small, the paper guarantees that the pair is close to exactly commuting operators, suggesting a controlled path to a shared eigenbasis and cheaper coordinate-wise dynamics. Add an optional numerical repair step that projects the learned pair toward a simultaneously diagonalizable pair.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace the dense output of selected linear projections with a two-sided magnitude threshold that emits zero for small values but preserves signed large values. Learn one positive threshold per projection, or optionally one threshold per output channel, so the network discovers where sparse events can be removed while retaining outlier information.
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Replace a single global preconditioner for a multi-penalty neural objective with additive corrections adapted to the joint kernels of the penalty Jacobians. The optimizer is designed to remain effective when individual penalty weights change independently, avoiding the severe conditioning degradation that occurs when a correction space misses a singleton or partial joint kernel.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Use two points with approximately equal minibatch loss to construct an ellipcenter: the intersection of the normal lines through the two points, where the normals are their gradients. The resulting update uses local curvature information in the span of two gradients and can be relaxed toward the current parameters or combined with momentum.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Use the paper's MBM-GP construction to predict input-dependent big-M constants for ReLU disjunctions during neural-network verification. Exact activation-bound optimization is performed only at a small subset of input points, while a Gaussian-process upper confidence bound supplies conservative bounds elsewhere, reducing verifier preprocessing and potentially tightening the MILP compared with one global worst-case constant.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace a conventional optimizer step by a three-phase cyclic update in which successive parameter blocks or gradient components are exposed to two low-noise phases and one high-noise, chemically driven phase. Treat the loss decrease as mechanical work, phase-dependent gradient-noise scales as reservoir temperatures, and an auxiliary drive as chemical free energy. Adapt the drive toward a target positive cycle affinity rather than increasing the learning rate indefinitely, creating a measurable…
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Replace the usual momentum state in an optimizer with a persistent Ornstein-Uhlenbeck-driven velocity subject to a dry-friction threshold. Correlated forcing can help traverse shallow noisy regions, while the friction term suppresses parameter motion when the effective force is small, potentially reducing update noise and improving late-stage stability.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Replace fixed-budget token or patch pruning with greedy selection that combines a teacher-derived relevance score and Gaussian-process mutual information. Select an item when it is both relevant and non-redundant, and stop when the largest remaining information gain falls below a calibrated threshold instead of retaining a fixed number of items.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace online enumeration over a finite action set with a classifier or lookup map whose regions directly return the action minimizing a one-step predictive-control cost. For affine dynamics and quadratic tracking loss, exact action regions are separated by pairwise cost boundaries, so the approximation can be audited against exhaustive predictive control rather than treated as an unconstrained policy.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace projected or retracted updates for constrained spherical parameter blocks with a geodesic Frank–Wolfe update and an iteration-only step size \(\eta_k=a/(k+a)\). The method moves along a minimizing geodesic toward a feasible linear-oracle point, avoiding repeated projection and eliminating line-search or gap-feedback overhead. On locally error-bounded objectives, the paper predicts accelerated polynomial convergence, including \(O(k^{-2})\) for strongly geodesically convex objectives…
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Approximate an expensive neural objective as a local second-order Hermite polynomial over a symmetric action stencil, then optimize the fitted polynomial rather than repeatedly evaluating the original objective. Unlike a Taylor model, the coefficients are obtained from function values and do not require reliable action derivatives through a simulator or learned environment.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Before or during graph-neural-network inference, use a tree dynamic program to choose a limited set of active computation nodes and upgraded message-passing edges. A node receives an embedding from an active landmark only when the selected path has enough residual communication radius, so the planner directly optimizes weighted coverage under a joint node-and-edge budget. The resulting active subgraph is then used by a sparse GNN or graph transformer.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Split a neural controller into a slow model-based planner and a fast policy instead of requiring either component to perform the entire control task. The MPC output provides a slowly varying nominal action or operating envelope, while the neural policy generates high-frequency residual corrections. This should preserve constraint handling while reducing the frequency of expensive online optimization.
Useful6/10
Difficulty5/10
Novelty6/10