Unverified
2026
Use the shared-marginal KL projection to turn token-to-expert routing into a low-rank, exactly balanced assignment rather than relying only on an auxiliary load-balancing penalty. Tokens retain normalized routing distributions while the shared latent marginal enforces consistent aggregate usage across two independently learned routing factors.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace count-only MoE load balancing with greedy balancing of aggregate token-feature vectors. A token is assigned to the expert for which adding its feature vector produces the smallest increase in that expert's squared aggregate norm, encouraging experts to receive complementary semantic mixtures rather than identical token counts.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace the ordinary update in a differentiable minimax game with a Halpern-anchored second-order operator step. The current game iterate is first corrected using the local Jacobian of the game gradient, and the corrected point is then contracted toward a fixed anchor with a decreasing Halpern weight. This is intended to reduce cycling in adversarial training while preserving the faster asymptotic behavior associated with second-order monotone-operator methods.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Grow a mixture-of-experts layer by splitting one expert into two children while conserving its routing mass, and choose the split ratio to minimize the worst imbalance over all intermediate expert counts. Use the paper's sharp threshold as a hard design target: with n experts, some intermediate stage must have capacity ratio at least D_n = 2^{1-1/\lceil n/2\rceil}, so schedules substantially below this are impossible rather than merely difficult to discover. Initialize child router logits with…
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Replace raw polynomial or Fourier-like features in a small MLP with basis functions orthonormal under a Sobolev inner product that jointly measures feature magnitude and input derivative magnitude. This explicitly controls feature smoothness while preserving decorrelation, potentially improving conditioning and reducing the need for large derivative-regularization coefficients.
Useful6/10
Difficulty4/10
Novelty7/10
Unverified
2026
Replace a continuous allocation or routing decision with a lattice-valued decision whose unit size is explicitly normalized by total capacity. Round allocations downward rather than to the nearest lattice point, preserving per-example capacity feasibility, and train or evaluate against the resulting granularity ratio rather than treating discretization as an implementation detail.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a Gaussian or point-estimate regression head with a heteroscedastic Student-t head whose scale and degrees of freedom depend on the learned state. This gives the model a principled way to absorb abrupt, nonmonotone events and operating-condition shifts without forcing the central degradation trend toward rare extreme residuals.
Useful6/10
Difficulty3/10
Novelty4/10
Unverified
2026
Use the Osgood transform as a controller for adaptive residual-layer step sizes. Instead of choosing a fixed residual scale or requiring every block to have a small operator norm, reduce the step only when the predicted transformed pairwise distance consumes too much regularity budget.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Add a finite-difference derivative branch to a neural feedback policy, but constrain its gain using the sampled-system fast-mode criterion from the paper. The controller can retain derivative information while avoiding high-frequency instability caused by the stored previous observation, especially when the control loop is sampled rapidly.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Represent each recurrent latent state as a pair of unit quaternions \((q_1,q_2)\in\mathrm{SU}(2)^2\), and evolve it with a composition of elementary Nielsen maps corresponding to a chosen hyperbolic matrix \(A\in\mathrm{SL}(2,\mathbb{Z})\). The layer exactly preserves the group manifold and Haar volume, preserves the commuting locus \(q_1q_2=q_2q_1\), and reproduces toral hyperbolic dynamics there, giving a structured long-horizon prior instead of an unconstrained matrix recurrence.
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Replace a learned dense bilinear map with a structured family of signed orthogonal matrices. Given feature vectors y,z in R^n, produce r interaction features h_a = y^T H_a z / sqrt(n), where the H_a form a Hadamard/Clifford-like family; the resulting bilinear map has operator norm at most one when r is within the Hurwitz–Radon limit. Learn only channel projections and optional scalar gates around this fixed mixer.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Augment an RNN or state-space layer with binary reversible gates: active units update normally, while paused units hold or weakly update their hidden state and temporarily suppress downstream activity. Tune the pause probability so that the expected number of paused units is near Np* ≈ 1.5, creating intermittent long-memory episodes without pausing the entire layer. The paper predicts that this regime should maximize low-frequency output variability and may improve tasks requiring rare…
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Replace the scalar learning rate of SGD or Adam's outer update by a blockwise Barzilai--Borwein estimate computed from consecutive parameters and gradients. Use gradient smoothing, denominator checks, and clipping so that the curvature estimate remains usable with stochastic neural-network gradients.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace backpropagated gradients in a Langevin sampler with a simultaneous-perturbation estimate obtained from two noisy evaluations of a neural energy or simulator-defined negative log-density. This enables posterior or latent-space sampling when the energy contains nondifferentiable code, stochastic simulation, discrete operations, or an inaccessible neural-network gradient.
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Use a symmetric graph stress matrix as the interaction operator in a residual GNN or recurrent message-passing block. Enforce negative semidefiniteness and a prescribed nullspace containing invariant modes, transferring the paper's stress interpretation into an explicit contraction and stability certificate.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Use a low-rank controller that observes and actuates only the graph's harmonic coordinates rather than all edge features. For a graph with first Betti number beta_1 = dim ker(B), a beta_1-dimensional cycle basis is sufficient to represent the entire harmonic sector, yielding a compact recurrent memory or adapter for circulation-dependent graph dynamics.
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Represent each trainable parameter block as a global scale multiplied by a normalized shape, and evolve the shape through a projected Hamiltonian optimizer. The optimizer is designed so that normalized weights can approach a stable central configuration while auxiliary momenta retain phase-space volume that prevents ordinary Hamiltonian dynamics from having a full-space attractor.
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Replace a conventional feature-pyramid sum by a bounded multiscale differential transform. At each scale, subtract a blockwise conditional expectation from a local average, then combine these residuals with bounded coefficients. Add a penalty on the largest interval response so that contributions from adjacent scales cannot accumulate destructively or explosively.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a learned dense token-mixing matrix or residual-state transition with a sparse diffusive mixer whose Laplacian has a deliberately small largest Jordan block. Balance the two chain lengths around the central coupling/core, because the paper proves that this minimizes the worst defective transient among the tridiagonal family. Use a scalar residual step size to move the non-consensus spectrum inside the unit disk while preserving the sparse structure.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Treat stochastic gradient training as motion in a random potential given by the neural-network loss, and use local curvature and barrier estimates to control injected Langevin noise. Instead of applying a fixed temperature, adapt the optimizer noise so that the observed escape rate from a basin matches a target rate predicted by thermal activation. This should reduce premature trapping in sharp minima while avoiding destabilization from excessive gradient noise.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Train a neural coefficient-recovery model with an additional loss that rewards observation sensitivity in every learnable coefficient direction. Instead of only minimizing the reconstruction error of the observed trajectory, explicitly discourage a nearly singular parameter-to-observation Jacobian, which should reduce ambiguous reconstructions and improve robustness to noise.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Construct an orthogonally equivariant residual map on symmetric feature matrices whose update is strongly monotone by adding the identity to a monotone isotropic tensor function. This provides a stability-controlled matrix block and a route to well-behaved inverse or fixed-point inference, rather than relying only on unconstrained residual weights.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace an unconstrained matrix nonlinearity on small symmetric feature blocks with the isotropic spectral lift of a permutation-equivariant monotone map on eigenvalues. The layer remains orthogonally equivariant, while the paper's equivalence transfers a scalar inner-product monotonicity certificate from eigenvalue space to the full matrix space.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Add a differentiable penalty to flow-matching batches that penalizes violations of the N-cyclic monotonicity inequalities implied by the minibatch OT reflow limit. The regularizer can either refine approximate Sinkhorn assignments or train the velocity field to preserve locally non-crossing endpoint geometry, providing a cheap alternative when exact assignment is too expensive.
Useful6/10
Difficulty4/10
Novelty6/10