△ Mechanism confirmed, baseline not beaten
2026
Replace a standard ReLU surrogate with an input convex neural network whose hidden-to-hidden weights are constrained to be nonnegative. The network remains piecewise linear and expressive, but its convexity allows downstream minimization to use continuous ReLU epigraph constraints instead of binary activation variables, potentially eliminating the integrality bottleneck of neural optimization.
Useful8/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Compress the matrix gradient or momentum before applying Muon's polar LMO, and maintain an error residual in the uncompressed gradient space. The residual prevents systematic sign quantization bias from accumulating, unlike error feedback applied after the nonlinear polar/sign operation. This is suitable for distributed training because workers communicate one sign bit per matrix entry while the server still applies a matrix-aware Muon step.
Useful8/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a deep feed-forward block by the fixed point z=phi(Wz+Vx+b), with the recurrent weight W constrained so that the fixed point is unique for every input. The same condition makes forward fixed-point iteration stable and makes implicit differentiation well-conditioned, allowing depth-independent memory usage while providing a measurable spectral failure boundary.
Useful8/10
Difficulty5/10
Novelty4/10
△ Mechanism confirmed, baseline not beaten
2026
Replace the unstable classical derivative of a discretized rough energy component with a matched dilation quotient derived from its intrinsic scale recursion. Use this field inside kick-drift-kick proposals and apply an exact Metropolis correction, allowing the proposal field to be measurable and nonconservative rather than an exact neural-energy gradient. The experiment should test whether acceptance rates and posterior samples remain stable as the rough-energy resolution increases.
Useful7/10
Difficulty5/10
Novelty8/10
△ Mechanism confirmed, baseline not beaten
2026
Add sparse directed coupling between parallel neural modules, recurrent states, or distributed replicas so that each module is driven toward a common trajectory without forcing an undirected or balanced communication graph. Select n-1 directed paths per strongly connected component and assign gains using the estimated Lipschitz bound of the uncoupled module; activate the coupling only when its graph-certified strength exceeds the predicted synchronization threshold.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace independent token-to-expert softmax routing with a fixed-budget congestion game. Each token group distributes a fixed routing mass across experts, while the marginal value of an expert decreases as other groups send mass there. Iteratively route toward the highest current marginal utility and exploit sorted-prefix supports to produce sparse, capacity-aware assignments.
Useful7/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace the standard diagonal or identity preconditioner used when solving an implicit neural layer with a coarse/fine Schur-complement preconditioner. The hidden state is decomposed into a low-dimensional coarse subspace and its orthogonal complement; the coarse interaction is solved accurately, while the fine block receives a damped approximate inverse. The method is especially suitable for deep equilibrium models, implicit MLPs, and Newton or quasi-Newton training of residual dynamics.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Partition a neural network into N interacting modules and constrain the Jacobian of its implicit residual map to be block diagonally dominant. Each module can compute its update locally while cross-module coupling is monitored through a normalized block-row margin. The certificate guarantees local nonsingularity of the equilibrium equations and predicts a sharp loss of robustness when the largest BDD ratio approaches one.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Use measured local Jacobian growth to set the variance of dropout, feature noise, or stochastic-depth perturbations, implementing the paper's fluctuation-response idea that multiplicative noise is tied to the positive scrambling or Lyapunov rate. The controller maintains a target growth regime instead of applying a fixed noise schedule throughout training. It predicts a stability transition when the estimated growth rate crosses zero and a variance-growth proportionality that can be tested…
Useful7/10
Difficulty4/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace synchronous replicated-gradient computation with a bounded-staleness stream: at optimizer step t, aggregate one gradient for each data partition, using the newest completed evaluation even if it was computed at an earlier model version. Replicated partition placement makes the aggregate robust to stragglers, while pipelining ensures that each worker computes only one partition gradient per step.
Useful7/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace star-shaped parameter synchronization with a rooted-tree primal-dual optimizer in which each worker owns a parameter block and communicates only with its parent and children. Dual updates performed at a node are explicitly redistributed as child correction messages, preventing stale-consensus errors caused by level-synchronous execution.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace repeated iterations of an expensive high-dimensional update S with iterations of a lower-dimensional latent map T, then decode the resulting latent state with D. Train E, D, and T with explicit intertwining losses so that encoding a full update agrees with updating the latent state, and decoding a latent update agrees with applying the original update.
Useful7/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Use small-gain diagnostics to jointly learn module normalization and a communication partition rather than imposing a fixed global spectral constraint. Clusters should be formed around high-gain feedback loops, because grouping weakly related modules cannot improve the certificate and only adds bookkeeping.
Useful7/10
Difficulty7/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Estimate an expensive fine-model trace or quadratic-form quantity using a telescoping sum over cheap-to-expensive neural approximations. Allocate many probes to cheap levels and only a few probes to the expensive level, exploiting strong correlation between adjacent levels to reduce variance at fixed compute. Candidate levels include truncated Transformer depth, reduced width, low-rank curvature, coarser graph resolution, or progressively tighter implicit-solver tolerances.
Useful7/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Represent a large positive semidefinite neural operator as the sum of two Kronecker products and regularize an efficiently computed upper bound on its largest eigenvalues. The bound controls not only the spectral norm but every top-k eigenvalue sum, allowing a tunable penalty on concentrated or unstable directions without constructing the exponentially larger operator.
Useful6/10
Difficulty5/10
Novelty7/10