A Refined Parameter Condition in the Lyapunov Analysis of IGAHD
arXiv:2608.28088
2026
Optimization
1 ideas extracted · analyzed Sep 2, 2026
What the math gives to ML
The paper gives a sharper Lyapunov-stability region for inertial gradient descent with Hessian-driven damping by retaining exact coefficients in a positive-definiteness argument. The transferable asset is a computable coupling between step size, smoothness estimate, and gradient-difference damping strength, which permits damping settings excluded by the older condition and remains valid for sufficiently small steps. A practical adaptation is an IGAHD optimizer with online smoothness estimation and a parameter controller that enforces the refined inequality while maximizing the admissible step size. The quadratic spectral analysis also supplies a useful diagnostic: on nearly quadratic problems, stable training should exhibit geometric decay rather than merely nonincreasing loss.
Ideas from this paper
Unverified
2026
Replace the momentum update in a gradient optimizer by inertial motion plus a gradient-difference term, which discretely approximates Hessian-driven damping. Choose the damping coefficient and step size using the paper's refined stability inequality instead of the older restrictive bound, and adapt them whenever the estimated smoothness changes.
Useful6/10
Difficulty4/10
Novelty5/10