Last-Iterate Convergence of Single-Loop Stochastic Methods for Constrained Convex-Concave Minimax Problems

What the math gives to ML

The paper identifies a concrete failure mode of stochastic saddle-point training: vanilla stochastic extragradient and optimistic gradient methods can keep oscillating or fail to converge in their final iterate even on bilinear objectives. Its transferable asset is a Tikhonov-style perturbation that makes the minimax objective strongly convex in the minimizing variables and strongly concave in the maximizing variables, converting noisy rotational dynamics into contractive dynamics around a unique regularized saddle. A practical neural-network analogue is to add opposite quadratic anchoring terms to adversarial objectives and train with stochastic extragradient. The fixed-horizon version is particularly easy to test because the perturbation strength can be selected from the planned training budget and evaluated using final-iterate loss, gradient norm, and game residual rather than trajectory averages.