Unverified
2026
Replace the single backpropagated subgradient of a piecewise-smooth network loss by a minimum-norm convex combination of gradients evaluated at nearby parameter perturbations. Shrink the perturbation radius geometrically and restart the schedule when the sampled Goldstein direction becomes small, following the paper's INGD motivation.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Approximate a dense symmetric interaction matrix in a neural layer by \(\widehat A=C\widehat M C^{\top}\), but compute the small core \(\widehat M\) from a two-sided sketched least-squares fit rather than from the landmark principal submatrix. This preserves signed or indefinite directions and avoids exploding outputs caused by an almost-singular \(A(I,I)\).
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a single plug-in top-k router decision with a confidence correspondence containing every router parameter candidate and sparse expert assignment that remains compatible with calibration and the current input. Project this set onto a hierarchy of expert groups and return the finest group-level decision supported by all surviving explanations; otherwise coarsen the route or abstain. Active endpoint bracketing evaluates only candidates that could still change the projected routing report.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Add two scalar adaptive gains to a neural controller or learned dynamical model: one estimates the unknown norm of the ideal neural approximation weights, and the other estimates the combined approximation, friction, and disturbance envelope. Sigma modification prevents unbounded gain growth, while the robust residual correction uses only these scalar estimates, independent of the number of neural features.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace feature-only graph pooling with a relaxed spectral-minimal partition layer. The layer assigns nodes to k clusters while favoring clusters with large algebraic connectivity, producing coarsened nodes that are internally well connected and less likely to contain bottlenecks. The resulting pooled graph can be used by a hierarchical GNN or graph transformer.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace an unconstrained dense transition or recurrent matrix with a normal matrix $A=U\operatorname{diag}(\lambda)U^*$, where $U$ is unitary and $\lambda$ contains learnable eigenvalues. The layer can be initialized by fitting a normal matrix to input-output pairs through the paper's objective, then trained with Riemannian updates that keep $U$ unitary and preserve the normal-operator structure.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace repeated substrings in long sequences with nonterminal symbols from an acyclic straight-line grammar, then run the transformer on the compressed sequence. Unlike ordinary fixed tokenization, the compression objective explicitly minimizes the number of reusable binary productions, allowing repeated document-specific or corpus-level motifs to become single units. An expansion map lets the model recover token-level outputs for selected positions.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Partition a long integration interval into M short segments and assign one neural trajectory approximator to each segment. Instead of asking a single network to satisfy the ODE and initial condition over the entire horizon, construct every segment so that its value at the left boundary is exactly the terminal value predicted by the previous segment. This removes interface discontinuities from the optimization problem and should improve long-horizon trajectory accuracy and gradient stability.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Represent hierarchical routing decisions by one weighted bipartite graph between parent experts or regions and fine cells or token groups. Select a regularized edge set once, then derive both coarse parent activation and fine-grained routing from it, preventing later refinement from invalidating earlier load balancing.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Train an MoE router against demand shifts in which only a limited number of token groups can simultaneously move to their maximum deviation from nominal frequencies. For each adversarial demand, compute a balanced routing equilibrium or approximate system-optimal dispatch, and penalize the resulting worst expert congestion rather than only the nominal batch load. This should make routing robust to correlated bursts while avoiding the excessive conservatism of perturbing every token group…
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Add a stochastic pairwise regularizer that penalizes only coordinate pairs whose normalized neural-field difference exceeds a threshold. Unlike a conventional fractional Sobolev penalty, the weak-type functional uses an indicator and a distance weight, and its Gamma-limit guarantees convergence toward a local gradient energy as the threshold grows.
Useful6/10
Difficulty4/10
Novelty7/10
Unverified
2026
Use active-basis changes as a cheap, solver-derived indicator that a policy update has crossed a nonsmooth decision boundary. Adapt the neural optimizer’s step size and gradient confidence using the fraction of trajectory decisions whose bases disagree between the current and proposed policy, preserving large steps in locally affine regions and damping updates near combinatorial switches.
Useful6/10
Difficulty4/10
Novelty7/10
Unverified
2026
Insert a projection step after a graph neural network proposes edge weights, replacing the proposed Laplacian by the closest valid Laplacian with a prescribed block-component structure. The projection removes cross-block interactions while minimally changing within-block weights, and a block spectral-gap constraint guarantees that each block is connected rather than accidentally splitting into smaller components.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace standard black-box hyperparameter search with a trust-region optimizer whose local quadratic surrogate includes an explicit linear dependence on wall-clock time or training-step age. Fit the model with ridge-regularized quadratic interpolation, then use a drift-compensated trust-region ratio to avoid rejecting useful moves merely because the validation distribution has deteriorated over time.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a fixed-temperature softmax router over experts, adapters, or candidate optimizers with an exponential-weights distribution whose temperature is selected to satisfy an explicit cumulative information budget. The router reacts strongly when observed expert losses are predictable, but automatically cools down when outcomes create a large cumulant-information gap, avoiding variance-based heuristics that can be badly miscalibrated. A prior distribution over experts supplies a principled…
Useful6/10
Difficulty5/10
Novelty4/10
Unverified
2026
Use a window of observed neural-network update trajectories to identify the set of local quadratic objectives and preconditioners that are consistent with the observed optimizer behavior. Rather than selecting one arbitrary curvature model, retain the nullspace of compatible parameters and use its dimension or smallest singular value as an identifiability and stability diagnostic.
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Attach a finite mixture of zonotopes to each uncertain neural input or hidden state, and propagate every mixture component through affine layers and conservative nonlinear relaxations. When the number of components grows, merge components only with an enclosing zonotope and sum their probability masses, preserving a formal lower bound on the probability that the true activation lies in the represented set.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace alternating descent/ascent with a single primal-dual Newton update for a constrained min-max neural-network objective. The optimizer maintains primal variables, equality multipliers, inequality slacks, and a barrier parameter, so the adversary remains feasible in the limit without hard projection and the coupled dependence of constraints on both players is represented in one linear system.
Useful6/10
Difficulty7/10
Novelty7/10
Unverified
2026
Represent an iterative neural computation as a controlled dynamical system and learn sparse residual corrections that are active only for a finite prefix of iterations. Estimate local stable and anti-stable subspaces of the hidden-state Jacobian, increase the correction horizon only while the anti-stable component exceeds a tolerance, and force later controls to zero. This produces adaptive-depth inference with a quantitative stopping criterion.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace unconstrained latent or neural-ODE dynamics with a strict-feedback cascade whose virtual controls are generated recursively by nonadaptive backstepping. Add a fixed internal-model oscillator when the desired output contains known-frequency periodic components, so the network tracks persistent targets without learning an unstable long-memory representation. The controller is designed to tolerate bounded neural-model mismatch and disturbances through an input-to-state stability margin.
Useful6/10
Difficulty7/10
Novelty7/10
Unverified
2026
Replace the usual first-order parameter update with controlled position-velocity dynamics. The loss is the potential energy, momentum is the velocity, and a one-step rolling-horizon control minimizes the predicted next-step energy plus a control penalty, producing an explicitly dissipative correction that can be applied only through a low-rank or blockwise control operator.
Useful6/10
Difficulty5/10
Novelty4/10
Unverified
2026
Replace overflow dropping in a mixture-of-experts layer with a discrete-convex load repair procedure. The router first chooses experts from neural logits, then applies capacity-aware exchange moves that preserve the total number of dispatched tokens and monotonically improve the routing objective whenever a feasible swap exists.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace an unconstrained multiplicative interaction between two nonnegative neural features by a lifted gate whose first and second moments satisfy the paper's semidefinite relaxation for the set F = {(x1,x2): x1,x2 >= 0, x1 x2 <= 1}. Insert the gate into an MLP, attention score, or MoE router to prevent explosive feature products while retaining a tractable convex feasible set.
Useful6/10
Difficulty7/10
Novelty8/10
Unverified
2026
Insert a few implicit DLSS diffusion steps after a network produces a nonnegative spatial probability field, such as a segmentation map, density estimate, or normalized image likelihood. The layer is a nonlinear fourth-order smoother that preserves positivity and is contractive in square-root/Hellinger distance, potentially reducing prediction noise without ordinary Euclidean blurring.
Useful6/10
Difficulty7/10
Novelty7/10