Strong error analysis for the stochastic momentum optimizer

What the math gives to ML

The paper derives an explicit second-moment recursion for stochastic momentum under a one-point convexity condition that controls both distance to a target and population-gradient magnitude. Its transferable asset is the decomposition of momentum energy into inherited momentum, a state-dependent term, and a minibatch-noise floor scaling as 1/J. This supports an online controller that adjusts momentum from estimated gradient noise and local alignment instead of using a fixed coefficient. The idea is most promising as a stability-oriented modification of SGD momentum or AdamW's first-moment coefficient.