Unverified
2026
Partition a neural network into heterogeneous parameter blocks or maintain several worker replicas, and model each block's optimizer state as a constrained linearized dynamical agent. At every synchronization interval, jointly optimize a finite sequence of parameter updates and a feasible common terminal parameter target, while enforcing consensus through distributed primal-dual iterations. Unlike ordinary gradient descent toward a fixed or implicit target, the target is selected together with…
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace decentralized parameter averaging with consensus on cumulative local gradient states, followed by a barrier-FTRL update that stays strictly inside a convex feasible set. This is particularly suitable for federated learning with heterogeneous clients and for simplex-constrained mixture, router, or adapter parameters, where Euclidean projection can be unstable or expensive.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Partition neural modules into two empirically identified reliability or noise classes and restrict their communication graph to a two-block stochastic block model. Allocate a fixed connectivity budget across within-class and cross-class edges using a water-filling update that favors block pairs producing the largest increase in validation utility. The resulting layer is sparse and modular, with a testable prediction that optimal connectivity concentrates on a few block pairs rather than…
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Use the paper's MBM-GP construction to predict input-dependent big-M constants for ReLU disjunctions during neural-network verification. Exact activation-bound optimization is performed only at a small subset of input points, while a Gaussian-process upper confidence bound supplies conservative bounds elsewhere, reducing verifier preprocessing and potentially tightening the MILP compared with one global worst-case constant.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Add a small dynamical state on the transformer module graph and use it to control adaptive computation, but reject controller parameters whose discrete-time update has latent roots outside the unit disk. The state can modulate halting thresholds, residual-block gains, and memory gates; the certificate applies to the controller integrator and prevents unstable oscillations or exploding internal control signals during long adaptive-depth rollouts.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Model dynamic routing as a multitype branching process: an active token of type d probabilistically creates child activations of type d'. Estimate the corresponding mean offspring operator and regulate its Perron root to a target reproduction rate, typically near one. This should make adaptive-depth or recursively routed networks use sparse computation without producing either rapidly vanishing paths or uncontrolled activation explosions.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Split a neural controller into a slow model-based planner and a fast policy instead of requiring either component to perform the entire control task. The MPC output provides a slowly varying nominal action or operating envelope, while the neural policy generates high-frequency residual corrections. This should preserve constraint handling while reducing the frequency of expensive online optimization.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Use SPINE's nested entropy profile on the singular values of each trainable weight matrix to discover spectral bands online, rather than choosing a fixed rank or a fixed number of learning-rate groups. Assign smaller step sizes or stronger decay to dominant singular-value bands and larger step sizes to weak bands, while updating the grouping only when the entropy-boundary signal is persistent.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Represent edge or pair-token features and propagate them with a convex mixture of two normalized channels: transitions through shared vertices and transitions through shared triangles. This preserves higher-order connectivity that an ordinary graph convolution loses, while the mixing coefficient q controls whether information follows pairwise support or genuine triangular structure.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Modify decentralized parameter averaging or graph message passing so that each communication edge is classified using its observed delay and the spectrum of the instantaneous communication graph. Fast edges retain cooperative coupling, while excessively stale edges are attenuated or treated as antagonistic in a signed-Laplacian stability test. This should prevent a small number of very stale links from destabilizing otherwise stable asynchronous training.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a dense neural interaction graph by a dynamically activated graph whose edge $(u,v)$ is retained only when its effective coupling exceeds the local spacing of response modes. The network remains sparse below the connectivity transition but becomes globally communicating once a giant component forms, providing a controllable alternative to arbitrary magnitude pruning.
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Replace a stationary federated optimizer with a decentralized optimizer whose target distribution explicitly forgets old streaming samples. Each round performs only K consensus-gradient iterations, with K selected from the mixing contraction so that the communication budget matches the temporal volatility of the objective. The method should react faster to distribution shifts while limiting disagreement and bias caused by heterogeneous clients.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace fixed-period federated averaging or distributed all-reduce with a Bernoulli communication decision whose probability is selected from estimated network connectivity and optimization conditioning. Local workers continue making corrected updates between communication events, while the contraction theorem exposes when communication is worth its cost.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Represent stochastic layer execution, branching, retries, and early exit as a finite continuous-time Markov chain, with the completed-prediction state absorbing. Learn transition rates jointly with neural-network weights, but use MFPT sensitivities to allocate rate changes according to their available control budget rather than allowing one routing edge to dominate halting-time control.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace uniform or purely loss-driven update allocation with a scheduler that targets both the mean update rate and the temporal variance of updates for each parameter group, task, or expert. At every training step, assign the available minibatch slots or accelerator workers to groups with the largest weighted deficits, preventing starvation while avoiding highly bursty update streams that can produce optimizer oscillations.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Add a periodic coarse optimization phase to SGD or Adam that operates on a compressed parameterization and returns a prolongated correction to the full network. Retain nonsmooth constraints or regularizers explicitly through a primal-dual update instead of relying on penalty smoothing. Accept the correction only when it improves a cheap fine-batch merit test, making the method useful even when the coarse objective is only approximately coherent with the fine objective.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace Euclidean projection onto a smooth convex parameter constraint by a local tangent-halfspace linear minimization step. The method is particularly suitable for convex softmax heads or low-dimensional adapter parameters constrained to an ellipsoid, where projections require matrix solves or expensive nonlinear routines but membership and boundary-normal queries are cheap.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Reparameterize all Bayesian neural-network weights by a bounded latent vector in the unit ball and use a simple ball-constrained MCMC kernel instead of unconstrained HMC or random-walk sampling. A radial diffeomorphism expands points near the ball boundary into arbitrarily large weights, preserving heavy-tailed posterior mass while preventing the sampler from numerically wandering through an unbounded parameter space.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace communicated floating-point gradients in synchronous federated or data-parallel training with one-bit threshold queries whose thresholds are sampled publicly before gradients are observed. Use several fixed geometric amplitude scales so the same protocol handles unknown gradient means and heavy-tailed client updates without an interactive localization round. Decode each coordinate from the scale whose neighboring estimates are statistically consistent, then apply the decoded aggregate…
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Attach a symbolic sparse head to a neural encoder instead of using a dense final MLP. The head evaluates a library of learnable power-law and interaction terms on nonnegative learned features, jointly optimizes linear coefficients and exponents, and removes inactive terms with coefficient sparsity. This should provide a compact model with better relative-error behavior on positive targets spanning several orders of magnitude.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Track the dominant rank-r subspace of the gradient covariance online, then use that basis to construct a low-rank adaptive update or a controlled preconditioner. Unlike offline PCA refreshes, the Oja flow continuously follows changing training geometry while preserving orthonormality, potentially reducing the cost of second-order or Shampoo-like methods.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a costly full-data conjugate-gradient solve for a neural-network linearized least-squares step with a sequence of progressively larger sketched solves. Each solve starts from the previous solution, so early iterations cheaply identify the useful update direction and only the final few iterations use the full training batch.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace ordinary sign-gradient or fixed-range gradient quantization in distributed training with a non-adaptive one-bit refinement block. Workers generate all localization and multiscale residue bits using public randomness before the server knows the current aggregate gradient; after decoding a coarse center, the server chooses which stored phase and scale interpretation to use. This should reduce clipping and variance for heavy-tailed gradients without requiring a second communication round.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace the usual independently normalized MoE router scores with a small system of tropical inequalities controlling expert load, score margins, and capacity slack. Each inequality induces a local max-plus policy selecting its currently dominant expert or constraint; policy improvement detects positive-weight cycles that would cause oscillatory routing and applies the smallest bias correction that removes them. This provides a non-differentiable but cheap controller around the router…
Useful6/10
Difficulty6/10
Novelty6/10