Variance-budgeted stochastic momentum

(1) Integrate the method into the momentum-buffer update of SGD, or replace AdamW's fixed first-moment coefficient β1 while leaving its second-moment estimator unchanged. Maintain a momentum buffer m_t, an exponential estimate of its squared norm, an estimate of minibatch gradient variance, and a short parameter displacement. (2) Pseudocode: compute two independent microbatch gradients g1 and g2; set g=(g1+g2)/2 and sigma_hat²=||g1-g2||²/2. Maintain M_hat=EMA(||m_t||²), displacement d=theta_t-theta_{t-K}, and D_hat=||d||²/(K²+epsilon). Estimate local alignment with c_hat=clip(abs(<d,g_t-g_{t-K}>)/(||d||²+epsilon),c_min,c_max). Set a target momentum-energy budget B_t=q(||g||²+epsilon). For each candidate alpha in a grid from alpha_min to alpha_max, compute B_hat(alpha) using the displayed adapted inequality and choose the largest candidate satisfying B_hat(alpha)≤B_t. Update m_{t+1}=alpha_t m_t+(1-alpha_t)g, then theta_{t+1}=theta_t-eta_t m_{t+1}. Smooth sigma_hat, c_hat, and D_hat with EMAs and use conservative clipping because the theorem's target point and convexity constant are not known. (3) The paper supplies the recursion and the 1/J noise scaling; all estimates of c, distance to the target, and the noise bound are empirical proxies. (4) First experiment: train ResNet-18 on CIFAR-10 and a small Transformer on WikiText-2, comparing fixed-momentum SGD and AdamW against the controller at identical batch size, learning-rate schedule, and update count. Track loss at equal FLOPs, gradient-noise estimates, momentum norm, divergence or overflow frequency, and final validation accuracy or perplexity. The expected signal is fewer loss spikes and fewer unstable runs during noisy early training, with equal or faster loss reduction and no late-training accuracy loss.