Unverified
2026
Replace an empirically chosen momentum or learning-rate modulation by a forcing amplitude calibrated to the homoclinic energy balance of a reduced optimizer mode. The controller deliberately operates below the separatrix-crossing threshold when stable refinement is desired, or slightly above it when the optimizer must escape a basin. This creates a falsifiable transition prediction rather than merely adding noise or tuning a schedule.
Useful5/10
Difficulty6/10
Novelty8/10
Unverified
2026
Train Fourier or state-space neural models by eliminating well-conditioned spectral modes first and retaining near-resonant modes until a later stage. The schedule is determined by the small-divisor geometry of a reference transport vector, with a cumulative Brjuno-like budget controlling how aggressively spectral corrections may be applied. This should prevent rare nearly resonant modes from producing disproportionately large gradients or unstable long-horizon rollouts.
Useful5/10
Difficulty5/10
Novelty8/10
Unverified
2026
Replace penalty-based equality-constrained training with a two-timescale optimizer. A fast variable tracks the normal correction that drives constraint residuals toward zero, while the slow parameter update follows the task gradient projected onto the local constraint tangent space. This should reduce sensitivity to very large penalty weights and preserve feasibility more accurately during training.
Useful5/10
Difficulty5/10
Novelty5/10
Unverified
2026
Use a Christoffel word as a periodic binary gate for an expensive training operation: activate the operation exactly r times in every N-step period, but distribute those activations as uniformly as possible rather than in blocks or independent Bernoulli trials. Candidate operations include SAM perturbation steps, Hessian-vector preconditioning, gradient clipping, EMA teacher refreshes, or an auxiliary MoE expert. The intended benefit is lower burst-induced gradient variance at the same average…
Useful5/10
Difficulty3/10
Novelty7/10
Unverified
2026
Use Oja's streaming eigenvector estimate on a parameter block's incoming gradient stream, but activate its rank-one preconditioning correction only after the mathematically predicted d log d sample threshold. Before that point, the estimate is treated as unreliable and the optimizer remains close to AdamW or SGD. This prevents early noisy spectral directions from destabilizing training while retaining an O(d)-memory alternative to storing a full gradient covariance matrix.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace ordinary projected-gradient updates for a convex neural subproblem with a homogeneous perspective formulation and Douglas-Rachford splitting. The additional scale variable makes the update less sensitive to large variations in loss or parameter scale and can expose infeasible combinations of constraints instead of producing unstable iterates.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Use the paper's effective operator 𝒢 = (I + K⁻¹L)⁻¹ as a learned, geometry-aware preconditioner for momentum or latent-state updates. The coupling matrix L changes the response of momentum variables without changing coordinate components, providing a controlled mechanism for mixing fast and slow latent channels.
Useful5/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace a fixed weighted sum of normalized neural-network objectives with a differentiable fuzzy scalarizer that assigns every criterion to desirable, tolerable, and undesirable regions. Explicit output consequents turn these semantic classes into a scalar training loss, while localized memberships reduce flat plateaus and make the optimizer distinguish genuine preference minima from arbitrary ties.
Useful5/10
Difficulty4/10
Novelty6/10
Unverified
2026
Use the cluster-state construction to schedule which groups of trainable parameters receive an expensive update at each optimizer micro-step. Instead of updating every LoRA block, expert group, or layer uniformly, select the block whose local error histogram predicts the largest loss reduction per unit compute.
Useful5/10
Difficulty6/10
Novelty6/10
Unverified
2026
Introduce an auxiliary matrix-valued optimizer state whose update is a Lie–Poisson flow discretized by similarity transforms rather than additive Euler steps. Because similarity transforms preserve $\operatorname{tr}(Z^k)$ and the full eigenvalue multiset, long training runs avoid spectral drift in the optimizer state; the state can then generate a preconditioned update for ordinary neural-network parameters.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace an unconstrained simplex router or differentiable mixture layer with a resource-cost-aware router whose learned costs satisfy the paper's monotonicity curvature condition. Use a Euclidean-regularized Frank–Wolfe oracle to update routing probabilities, which should reduce cycling and sensitivity when several examples or agents compete for the same experts.
Useful5/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace or supplement spectral-norm and Frobenius penalties on neural-network weight matrices with the Hardy-type norm given by the geometric mean of their gains over uniformly sampled unit directions. This penalizes typical multiplicative amplification through a logarithmic average, while the paper's theorem guarantees that the resulting quantity is a true norm rather than an ad hoc nonconvex statistic.
Useful5/10
Difficulty3/10
Novelty6/10
Unverified
2026
Replace purely deterministic training trajectories with an optimizer that periodically resets parameters to a reference checkpoint at iid random renewal times. Use the renewal equation to compare how different reset-time distributions trade off uninterrupted progress against recovery from poor regions, and trigger resets when the observed loss trajectory matches the predicted low-progress regime.
Useful5/10
Difficulty4/10
Novelty5/10
Unverified
2026
Periodically project a rectangular neural-network weight matrix onto an approximately orthonormal-column matrix using LU-preconditioned CholeskyQR rather than ordinary QR or a polar iteration. Pivoted LU handles badly scaled and nearly dependent columns, while Householder orthogonalization of the LU factor produces a triangular preconditioner that makes the subsequent Cholesky step safer in fp16 or bfloat16.
Useful5/10
Difficulty6/10
Novelty5/10
Unverified
2026
Use a two-gradient predictor-corrector average as the gradient supplied to Adam, retaining trajectory smoothing while avoiding the three or four gradient evaluations required by full RK3. Vary the mixing coefficient to test whether the reported regularization comes from gradient averaging itself rather than from high-order integration.
Useful5/10
Difficulty4/10
Novelty5/10
Unverified
2026
Maintain a small population of neural-network parameter replicas and interleave ordinary gradient steps with Boltzmann/Kac-style binary collisions. Each collision preserves the pair's mean parameter vector and relative-distance norm while randomly rotating the relative direction, with collision frequency proportional to a regularized negative power of replica distance.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Maintain an ensemble of neural-network parameter vectors, evolve each member for a fixed number of stochastic-gradient steps, then remove members with poor validation scores and resample survivors with replacement. This transfers the paper's repeated density intervention while leaving each member's underlying optimizer dynamics unchanged. In reinforcement learning, the same mechanism can duplicate high-return policies and produce an effective drift toward better policies.
Useful5/10
Difficulty5/10
Novelty2/10
Unverified
2026
Treat a scalar training control, such as task-mixture weight, weight decay, or sparsity penalty, as a parameter ramped through a sharp optimization transition. If the model starts from a highly correlated pretrained or partially trained state, compensate for the predicted marginal logarithmic memory by slowing the ramp according to a fitted logarithmic factor rather than using a pure power-law schedule.
Useful5/10
Difficulty5/10
Novelty8/10
Unverified
2026
Build a variational quantum neural network whose trainable 2-qubit Hamiltonian is projected into the Lee-Yang coupling cone and augmented by a uniform field term -h sum_i Z_i. The theorem certifies a nondegenerate ground state and a gap at least h/4, enabling imaginary-time state-preparation layers with predictable exponential suppression of excited-state error.
Useful5/10
Difficulty6/10
Novelty9/10
Unverified
2026
Apply consensus-based derivative-free optimization independently in parameter blocks that are expected to contribute additively to the objective, using noise projected into each block rather than isotropic noise over all parameters. The method is most suitable for low-dimensional trainable objects such as LoRA adapters, soft prompts, calibration vectors, or neural architecture hyperparameters, where maintaining a small population of particles is feasible.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
For a recurrent, state-space, implicit, or complex-valued neural network, partition the local input-output Jacobian into amplitude and phase channels and penalize excessive sensitivity in either channel. This transfers the paper's voltage-source stiffness mechanism to feature magnitude and phase, producing a stability monitor that can distinguish harmless amplitude sensitivity from destructive phase rotation.
Useful5/10
Difficulty5/10
Novelty6/10
Unverified
2026
Estimate the entropy production of short parameter-update trajectories by comparing the probability of the observed optimizer path with the probability of its time reversal. Use the estimate as an online signal to reduce the learning rate or optimizer noise when training becomes excessively irreversible, and optionally add a soft penalty to the training objective. This directly operationalizes the paper's Onsager–Machlup/path-probability construction without requiring a tractable global…
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Build an implicit layer from a piecewise-linear maximal monotone operator on visible variables z_* and auxiliary variables z_**, then eliminate the auxiliary block rather than exposing it in the network output. Compute the layer through a fixed point of the eliminated component of a nonexpansive resolvent, with damping when the auxiliary map is not strictly contractive.
Useful5/10
Difficulty6/10
Novelty6/10
Unverified
2026
Represent selected activations, weights, or optimizer accumulators as four floating-point limbs and evaluate products through tensor-core matrix multiplications encoding limb convolution. Retain the convolution components during reductions and renormalize only at block boundaries, avoiding branch-heavy multi-double arithmetic inside every multiply-add.
Useful5/10
Difficulty6/10
Novelty6/10