Unverified
2026
Use the determinant and trace-power identities of the rules matrix as a spectral diagnostic for recurrent or state-space training. Penalize unstable or excessively resonant modes through a truncated log-zeta objective, while retaining selected eigenvalues near the unit circle when long memory is desired. This gives a falsifiable transition criterion based on closed-walk growth rather than only gradient norms.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Apply the paper's empirical preimage-entropy construction to a learned recurrent transition map, penalizing excessive distinguishable hidden-state histories that produce the same current state while preserving multiple histories when the task requires genuine multimodality. Unlike a raw inverse-Jacobian penalty, the regularizer is computed only among inverse trajectories having similar empirical state distributions, so it distinguishes useful multimodal memory from uncontrolled branch explosion.
Useful6/10
Difficulty7/10
Novelty9/10
Unverified
2026
Replace an unrolled constrained inner optimization in a meta-learning or hyperparameter-learning system with a KKT-based single-level layer. Instead of imposing primal-dual complementarity exactly from the first iteration, solve a sequence of relaxed problems with decreasing complementarity tolerances, making early optimization smoother and reducing failures caused by degenerate active-set geometry.
Useful6/10
Difficulty6/10
Novelty4/10
Unverified
2026
Replace unconstrained token-to-expert routing with a nonnegative mixture of a small set of feasible routing or communication patterns. Enforce resource limits using the paper's one-sided positive-contribution bound, which gives a conservative certificate without enumerating all joint token activation scenarios. Expand the pattern set only when a separation procedure finds a routing pattern that improves the router objective while adding useful capacity information.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Add a recurrent associative matrix to each selected transformer layer so recent key-value relationships can be retrieved without retaining every past token or performing gradient updates. The matrix uses input-dependent retention and write gates, but retrieval is always performed from the pre-write state, preventing the current target from leaking into its own prediction. Frobenius-norm clipping makes the recurrent memory bounded and provides a direct stability control.
Useful6/10
Difficulty4/10
Novelty4/10
Unverified
2026
Partition a neural network into heterogeneous parameter blocks or maintain several worker replicas, and model each block's optimizer state as a constrained linearized dynamical agent. At every synchronization interval, jointly optimize a finite sequence of parameter updates and a feasible common terminal parameter target, while enforcing consensus through distributed primal-dual iterations. Unlike ordinary gradient descent toward a fixed or implicit target, the target is selected together with…
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace the noisy Hutchinson estimate of a neural-network Hessian trace with a variance-reduced Hutch++ estimate computed only from Hessian-vector products. Use the estimated normalized curvature to cap or rescale the optimizer step, so learning-rate reductions occur when the loss landscape becomes globally sharp rather than when an individual minibatch gradient happens to be large.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace the momentum update in a gradient optimizer by inertial motion plus a gradient-difference term, which discretely approximates Hessian-driven damping. Choose the damping coefficient and step size using the paper's refined stability inequality instead of the older restrictive bound, and adapt them whenever the estimated smoothness changes.
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Modify a diffusion sampler over a complete scenario trajectory z by exponentially tilting its prior toward a learned failure score s(z), rather than conditioning independently on environment and execution variables. The resulting sampler spends more evaluations in rare failure regions while preserving an explicit importance weight for estimating probabilities under the original distribution.
Useful6/10
Difficulty5/10
Novelty4/10
Unverified
2026
Replace a neural controller's pointwise action outputs over a finite horizon with Bernstein control points whose convex hull satisfies actuator and trajectory constraints. The network predicts the control points, while a robust margin accounts for bounded tracking or model-prediction error, making continuous-time actuator feasibility checkable from finitely many inequalities.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Use two bounded self-adjoint transition operators in a recurrent or state-space block and penalize their normalized Hilbert--Schmidt commutator. When the penalty is small, the paper guarantees that the pair is close to exactly commuting operators, suggesting a controlled path to a shared eigenbasis and cheaper coordinate-wise dynamics. Add an optional numerical repair step that projects the learned pair toward a simultaneously diagonalizable pair.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Add a Scaled-NAP auxiliary loss to a graph neural network whose node embeddings are converted into soft community or pooling assignments. The exponent alpha controls the resolution of the learned partition: alpha=0 favors locally persistent, fine-grained groups, while alpha=1 recovers modularity and its preference for larger volume-balanced groups.
Useful6/10
Difficulty4/10
Novelty7/10
Unverified
2026
Construct a generative or recurrent neural architecture with several contractive or mildly expanding branches, and explicitly control the geometric complexity of its invariant set using the sub-additive singular-value pressure of branch-Jacobian products. Instead of regularizing only the operator norm, the model can preserve anisotropic directions while targeting a desired attractor dimension, potentially improving coverage of structured data without uncontrolled folding or collapse.
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Replace the dense output of selected linear projections with a two-sided magnitude threshold that emits zero for small values but preserves signed large values. Learn one positive threshold per projection, or optionally one threshold per output channel, so the network discovers where sparse events can be removed while retaining outlier information.
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Replace fixed-beta RMSprop with a bias-corrected, slowly increasing-beta schedule. The paper’s error decomposition contains a stochastic-approximation term of order gamma_n and a memory term of order (1-beta)^2; setting (1-beta_n)^2 proportional to the current learning rate makes the memory contribution decay at the same scale as the optimization error instead of leaving a fixed residual.
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Replace decentralized parameter averaging with consensus on cumulative local gradient states, followed by a barrier-FTRL update that stays strictly inside a convex feasible set. This is particularly suitable for federated learning with heterogeneous clients and for simplex-constrained mixture, router, or adapter parameters, where Euclidean projection can be unstable or expensive.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a single global preconditioner for a multi-penalty neural objective with additive corrections adapted to the joint kernels of the penalty Jacobians. The optimizer is designed to remain effective when individual penalty weights change independently, avoiding the severe conditioning degradation that occurs when a correction space misses a singleton or partial joint kernel.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Use one fewer point than the exact support threshold for a weighted multi-output linear head. The paper proves that the worst-case multiplicative loss penalty at this budget is only 1+1/(dm^2), giving a principled memory-saving option rather than an arbitrary subset-size heuristic.
Useful6/10
Difficulty4/10
Novelty4/10
Unverified
2026
Insert a differentiable reaction-diffusion layer that converts dense token or pixel features into sparse, spatially coherent routing masks. Two competing orientations form complexes through conserved monomer reservoirs, so local assignments can cluster while opposite assignments mutually exclude one another instead of independently activating at the same location. The layer can be used as a soft-to-hard MoE router, attention-mask generator, or object-part grouping module.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace ordinary parameter updates for a constrained neural network with an annealed reciprocal-manifold flow. Each differentiable inequality constraint remains strictly satisfied during the optimization trajectory, avoiding projection or a per-step quadratic program. This is most useful for safety-critical policy learning, bounded network outputs, parameter-budget constraints, or training with explicit robustness inequalities.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Construct a neural state-space model with an explicit first-order input-to-output path instead of forcing every output to depend only on deeply propagated hidden states. Penalize or reject learned linearizations whose transfer matrix has relative degree greater than one, then train a storage-function certificate for the remaining passive dynamics. This preserves the paper's relative-degree compatibility condition while allowing high-order internal memory.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Use two points with approximately equal minibatch loss to construct an ellipcenter: the intersection of the normal lines through the two points, where the normals are their gradients. The resulting update uses local curvature information in the span of two gradients and can be relaxed toward the current parameters or combined with momentum.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Build actuator or parameter constraints directly into the neural controller using a differentiable hard-saturation map rather than penalizing violations after the fact. This makes the Lyapunov certificate apply to the actual bounded controller and prevents training from exploiting unrealistically large actions.
Useful6/10
Difficulty3/10
Novelty4/10
Unverified
2026
Extend regularized extragradient with proximal operators so nonsmooth penalties such as group sparsity, nuclear norms, or parameter constraints are applied at both prediction and correction stages. This can produce sparse or low-rank adapters while retaining the look-ahead stabilization for the smooth inner residual.
Useful6/10
Difficulty5/10
Novelty7/10