Heavy-Tailed First-Order Optimization for Polyak-Łojasiewicz Condition: High-Dimensional Minimax Bounds, High-Probability Guarantee, and Fixed-Dimensional Improvements

What the math gives to ML

The paper gives a principled route for training under finite-α-moment gradient noise, where variance may be infinite and conventional sub-Gaussian concentration or unclipped SGD guarantees fail. Its transferable asset is centered clipping combined with mirror descent: clipping controls rare, arbitrarily large stochastic gradients while preserving the common directional component of the gradients. A practical neural-network adaptation is a robust optimizer that estimates a local gradient center from a minibatch, clips residuals around that center, and performs an Adam-like or mirror-descent update. The main falsifiable target is improved tail robustness and convergence when gradients are contaminated by heavy-tailed noise.