Unverified
2026
Use a two-gradient predictor-corrector average as the gradient supplied to Adam, retaining trajectory smoothing while avoiding the three or four gradient evaluations required by full RK3. Vary the mixing coefficient to test whether the reported regularization comes from gradient averaging itself rather than from high-order integration.
Useful5/10
Difficulty4/10
Novelty5/10
Unverified
2026
Replace a dense or irregular binary interaction matrix in a value-aggregation layer with a signed sum of blocky masks. Each blocky mask groups a set of query rows with a disjoint set of key columns, allowing all queries in a group to reuse one summed value vector. This is most suitable for linear attention, graph message passing, or any layer where the interaction matrix is applied directly to values rather than passed through a row-wise softmax.
Useful5/10
Difficulty7/10
Novelty7/10
Unverified
2026
Replace an unconstrained bilinear matrix fusion or covariance head with \(\Phi(A,B)=\sum_{r=1}^R V_r^*(A\otimes B)V_r\). The output is PSD by construction, and the stronger block-level property makes the layer compatible with minibatches, mixtures, and Gram-matrix inputs rather than merely preserving positivity pointwise.
Useful5/10
Difficulty5/10
Novelty5/10
Unverified
2026
Train a neural queue or event-sequence predictor using trajectories generated under an exponentially tilted scheduled-arrival law that makes rare overloads common. Reweight each tilted trajectory by its likelihood ratio, while optionally oversampling the rare-event subset to improve prediction of tail behavior.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Use the paper's affine variation law to warm-start training across nearby constraint or conditioning levels. Instead of independently learning models for every level parameter, predict the change in the relative representation or loss from a structured Chern-form slope and optimize only the correction.
Useful5/10
Difficulty5/10
Novelty8/10
Unverified
2026
Compute a positive nonlinear torsion function on each input graph and append it to node features or use it to gate message passing. Unlike degree or ordinary Laplacian coordinates, the p-torsion field measures response to a uniform source and can expose global distance-to-boundary and bottleneck structure in a single scalar channel.
Useful5/10
Difficulty4/10
Novelty6/10
Unverified
2026
Apply consensus-based derivative-free optimization independently in parameter blocks that are expected to contribute additively to the objective, using noise projected into each block rather than isotropic noise over all parameters. The method is most suitable for low-dimensional trainable objects such as LoRA adapters, soft prompts, calibration vectors, or neural architecture hyperparameters, where maintaining a small population of particles is feasible.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Add a rigidity-based regularizer to a neural graph or point-cloud encoder whose output coordinates are constrained by selected pairwise distances. The regularizer detects infinitesimal edge-length-preserving motions using the rigidity matrix, then uses equilibrium stresses to penalize deformation directions that survive at first order but are not blocked at second order. This targets representation collapse and locally ambiguous geometric embeddings.
Useful5/10
Difficulty6/10
Novelty8/10
Unverified
2026
Replace a binary neural connectivity mask by independent Bernoulli edge probabilities and optimize a deterministic expected message-passing objective before discretization. The resulting module can search sparse GNN edges or expert-to-token routes without repeatedly sampling many discrete architectures during training.
Useful5/10
Difficulty5/10
Novelty5/10
Unverified
2026
Regularize a neural network using exact finite-difference interaction terms at a chosen perturbation scale, while retaining the covering decomposition of a composition f∘g. Instead of penalizing only the total mixed difference, separately penalize selected covering terms containing large subsets or overlapping subsets, which targets higher-order and nonlocal interactions without computing Hessians.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace softmax attention or dense MoE routing with a normalized Rayleigh–Jeans distribution over tokens or experts. If an item's energy is close to the chemical potential, its probability becomes disproportionately large, creating controllable low-energy condensation instead of the exponentially smooth allocation produced by softmax.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Build an implicit layer from a piecewise-linear maximal monotone operator on visible variables z_* and auxiliary variables z_**, then eliminate the auxiliary block rather than exposing it in the network output. Compute the layer through a fixed point of the eliminated component of a nonexpansive resolvent, with damping when the auxiliary map is not strictly contractive.
Useful5/10
Difficulty6/10
Novelty6/10
Unverified
2026
Add a two-channel geometric head producing scalar fields u(x) and v(x) on a two-dimensional input or latent coordinate domain. Train it initially with a moderate p-harmonic duality constraint, then anneal p upward so u approaches an infinity-harmonic field while v remains its rotated-gradient dual; this penalizes isolated steep gradient spikes and promotes smooth, coherent level sets.
Useful5/10
Difficulty5/10
Novelty6/10
Unverified
2026
Apply the paper's reversed weighted interaction inequality to two nonnegative feature maps generated from different augmentations or network branches. Maximizing the normalized nonlocal interaction should discourage collapsed or overly concentrated spatial representations while remaining invariant to overall feature amplitude.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Replace independent per-task fine-tuning directions with a learned connection that transports shared network weights across a low-dimensional task or domain coordinate space. Penalize connection curvature so that adapting from task A to task C directly agrees with adapting through intermediate task B, reducing order-dependent drift and improving interpolation between sparsely observed tasks.
Useful5/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace uniform set or point-cloud pooling with a microscopic weighting computed from pairwise feature-space distances. The resulting signed pooling vector should retain boundary and geometrically isolated points that ordinary mean pooling suppresses, potentially improving recognition when class information is concentrated on shape extremities or rare local configurations.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a neural layer's dense parameters by a real parameter tensor and a binary activity mask, then update the mask using importance scores measured across multiple perturbation intervals. Gates that repeatedly occur in non-dominated accuracy–cost configurations receive larger scores and are more likely to be retained or activated, avoiding uniformly random sparse rewiring.
Useful5/10
Difficulty5/10
Novelty4/10
Unverified
2026
Use the paper's three-periodic linear-gradient construction as a cheap online detector for Adam's failure mode. When recent gradients exhibit the pattern \((c,-1,-1)\), with \(c>2\), and the adaptive update repeatedly moves in a harmful direction, freeze Adam's normalization and use a short SGD or AMSGrad fallback before returning to Adam.
Useful5/10
Difficulty4/10
Novelty4/10
Unverified
2026
Add a fractional oscillation penalty to scalar functions produced by a neural network on an ordered grid. Unlike a derivative penalty, this remains meaningful for nonsmooth or nowhere-differentiable outputs and interpolates between total-variation-like behavior and Sobolev-like smoothness.
Useful5/10
Difficulty3/10
Novelty6/10
Unverified
2026
Insert a rearrangement operation on scalar feature maps sampled along an ordered coordinate such as time, spatial position, or a neural-field input grid. The operation sorts values into non-increasing order, preserving the empirical histogram exactly while provably not increasing the Riesz fractional variation in the ideal one-dimensional continuous setting.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Train a neural function under hard pointwise constraints by projecting its desired output-space update into the Clarke tangent cone of the admissible set at every sampled input. Fit the resulting feasible measurable direction with a parameter update instead of repeatedly allowing the network to violate constraints and repairing it with a penalty.
Useful5/10
Difficulty6/10
Novelty6/10
Unverified
2026
Add a Vandermonde conditioning objective to a mixture-of-experts router so that experts acquire distinct scalar routing signatures instead of collapsing onto the same score region. The regularizer uses powers of one learned scalar score and directly penalizes near-coincident expert scores, providing a finite-mode identifiability signal complementary to load balancing.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Add a differentiable penalty to a graph generator or graph predictor when its soft higher-order clique density violates the sharp lower bound implied by its lower-order clique density. The regularizer encourages generated graphs to have mathematically consistent motif statistics without hard-discretizing the predicted adjacency matrix.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Prepend an adaptive Savitzky-Golay derivative bank to a temporal neural network. For each input channel and derivative order, select the local window by minimizing Stein's unbiased risk estimate, then concatenate the raw signal with the estimated derivatives. This supplies denoised velocity and acceleration features without requiring clean derivative targets or forcing the backbone to learn unstable finite-difference filters.
Useful5/10
Difficulty3/10
Novelty6/10