Every idea extracted from recent arXiv mathematics papers — verified and unverified. Click an idea to open its full card; badges show the empirical verdict.
Replace the black-box equilibrium denoiser in an image-restoration DEQ with a positivity-preserving mirror-descent equilibrium driven by the exact Gamma likelihood and a discretized surface-area/mean-curvature regularizer. The equilibrium layer has a small number of learned scalar or channel-wise parameters instead of a large implicit CNN, while the exponentiated update prevents negative intensities and naturally matches multiplicative noise.
Compress only the difference between the current client gradient and a persistent control variate, rather than compressing the full gradient. As the control variate tracks the client gradient, the residual shrinks and the same communication budget produces less compression noise than direct gradient quantization.
Replace deterministic LoRA importance scores with one-sided tests of whether each rank-one update has population contribution at least a user-selected threshold. Maintain empirical contribution samples during fine-tuning, estimate their uncertainty, and prune the components with the weakest statistical evidence while respecting the target rank budget. The method should avoid deleting components merely because their latest minibatch gradient was small.
Represent a block of candidate neural updates or adapter components by symmetric influence matrices and select one sign for each component so their aggregate spectral effect is small. This imports matrix discrepancy into low-rank adapters, expert aggregation, or structured quantization, where controlling the worst direction of interference may be more useful than minimizing entrywise error.
Replace communicated floating-point gradients in synchronous federated or data-parallel training with one-bit threshold queries whose thresholds are sampled publicly before gradients are observed. Use several fixed geometric amplitude scales so the same protocol handles unknown gradient means and heavy-tailed client updates without an interactive localization round. Decode each coordinate from the scale whose neighboring estimates are statistically consistent, then apply the decoded aggregate…
Select the coordinates of a sparse adapter or sparse fine-tuning mask using both Fisher width and inverse-Fisher width. The mask should avoid parameter subsets that are cheap in the Fisher geometry but extremely large in the inverse-Fisher geometry, or vice versa, thereby controlling both prediction sensitivity and estimator-like uncertainty.
Replace ordinary sign-gradient or fixed-range gradient quantization in distributed training with a non-adaptive one-bit refinement block. Workers generate all localization and multiscale residue bits using public randomness before the server knows the current aggregate gradient; after decoding a coarse center, the server chooses which stored phase and scale interpretation to use. This should reduce clipping and variance for heavy-tailed gradients without requiring a second communication round.
Use stochastic Frank–Wolfe to train a neural submodule whose parameters lie in a convex feasible region without expensive Euclidean projection. The entropic robust objective supplies the stochastic gradient, while a linear minimization oracle enforces constraints such as simplex mixture weights, an l1 budget, or bounded adapter coefficients.
Replace repeated full-curvature factorizations when evaluating many Bayesian neural-network hyperparameter settings with one shared low-rank approximation of the data-induced precision update. Reuse the same curvature subspace while changing prior precision, observation-noise precision, or layerwise regularization, evaluating each candidate through a small rank-r matrix and Woodbury solves.
Add asynchronous nonlinear consensus steps between local-SGD updates in a federated or decentralized system. Instead of averaging a participating client with all peers, replace one client parameter vector by the coordinatewise p-mean of neighboring replicas, allowing sparse communication and reduced sensitivity to atypical client models.