Towards joint scaling laws with optimal batch size schedules

What the math gives to ML

The paper supplies an explicit sequence-to-sequence upper bound in which each learning-rate step is weighted by its future learning-rate tail and by the second moment of the stochastic gradient. This exposes a concrete control variable that is usually ignored: batch size changes the gradient second moment through variance reduction, so the bound can be optimized jointly over learning-rate and batch schedules. A useful neural-network transfer is to allocate a fixed example-compute budget across steps in proportion to the square root of the bound coefficient times the estimated gradient-noise scale. The derivation is convex under a standard inverse-batch variance model, while the main empirical question is whether this schedule improves loss descent or time-to-target in nonconvex Transformer training.