Forgetting-Factor Regret for Online Zero-Sum Games
arXiv:2607.07078
2026
Dynamics
1 ideas extracted · analyzed Aug 30, 2026
What the math gives to ML
The paper introduces an exponentially time-weighted saddle-gap criterion for nonstationary zero-sum games, replacing uniform historical regret with a metric that emphasizes current equilibrium-tracking quality. This is directly relevant to GANs, adversarial training, and other min-max neural systems whose data distribution, opponent, or objective changes during training. The practical transfer is a discounted saddle-gap monitor coupled to learning-rate and momentum control: react to recent tracking failure rather than to a long-window average that can hide instability. The method is lightweight, requires only approximate best-response evaluations, and is falsifiable against standard fixed-schedule min-max training.
Ideas from this paper
Unverified
2026
Track an exponentially discounted approximation to the current min-max saddle gap and use it to control the optimizer of a GAN or adversarial learner. If the recent gap rises, reduce both players' step sizes and clear stale momentum; if it falls consistently, cautiously increase the step sizes. Unlike ordinary loss EMAs, this signal measures whether each player is close to a recent best response and can detect equilibrium-tracking failure even when generator and discriminator losses look benign.
Useful6/10
Difficulty4/10
Novelty7/10