A Few Accelerated Algorithms for Convex Optimization under $(H_0,H_1)$-Smoothness

What the math gives to ML

The paper introduces a curvature model in which the Hessian norm is allowed to grow with the current objective gap, \(\|\nabla^2 f(x)\|\le H_0+H_1(f(x)-f^*)\), rather than being globally bounded by a single smoothness constant. This suggests an optimizer whose effective learning rate is conservative when the model is far from a solution and automatically becomes more aggressive as the gap shrinks. The transferable asset is not merely acceleration, but phase-wise adaptation to a quantitatively shrinking curvature envelope, combined with restarts and inexpensive inexact estimates. The cleanest first deployment is on convex neural-network subproblems such as the final linear head, embeddings, or LoRA factors, followed by tests on full-network training with safeguarded backtracking.