A Passivity-Based Analysis of First-Order Momentum-Based Methods

What the math gives to ML

The paper offers a transferable passivity certificate for momentum-based optimization: a sector-bounded gradient map becomes output strictly passive after a feedthrough loop transformation. For an L-smooth objective, subtracting D times the gradient from the input produces a strictness margin of 1/L minus D, requiring 0 < D < 1/L. This can be converted into a momentum optimizer with an explicit learning-rate ceiling and an online passivity monitor that detects when curvature, momentum, or stochastic noise destroys the certificate. The strongest implementation is a passivity-governed momentum schedule whose predicted transition occurs at D = 1/L, rather than tuning momentum only by benchmark performance.