Critical-Batch Momentum Scaling

Integrate the method into the optimizer learning-rate scheduler immediately before each parameter update. Inputs are the current batch size B, reference batch B0, base learning rate eta0, reference momentum rho0, current momentum rho, optimizer type, and safety factors cP and cN. For Polyak momentum, maintain a velocity v and use v_t=rho v_(t-1)+g_t followed by theta_(t+1)=theta_t-eta_P v_t. For Nesterov, evaluate the gradient at a look-ahead point theta_t-eta_N rho v_(t-1), then use v_t=rho v_(t-1)+g_t and theta_(t+1)=theta_t-eta_N v_t. Compute the scheduled rate with scale=min(1,cP*(B/B0)*((1-rho)/(1-rho0))) for Polyak, or scale=min(1,cN*(B/B0)^beta_hat*((1-rho)/(1-rho0))) for Nesterov, and set eta=eta0*scale. Start with cP=cN=0.5. Monitor loss, gradient norm, update norm, and non-finite values over windows of 50 to 200 steps. If loss or gradient norms grow persistently, halve the relevant safety factor; if the run remains stable for several windows, increase it by 10 percent. Estimate beta_hat from short Nesterov sweeps at batch ratios q in {2,4,8}: find the largest non-divergent eta at each ratio, then fit the slope of log(eta_crit) against log(q), clipped to [1,2]. The first experiment should use a small Transformer on WikiText-103 or C4 with a fixed token budget, comparing AdamW, fixed-rate SGD momentum, fixed-rate Nesterov, and this controller at B/B0 in {1,2,4,8,16}. Report maximum stable learning rate, validation loss at equal token counts, final one-pass loss, and wall-clock throughput. Success means stable operation at a larger batch with equal or lower iso-token perplexity and faster loss reduction per wall-clock hour.