The Fourth Quadrant: A Stylized View of Benign Misfitting

What the math gives to ML

The paper gives an exact example where interpolation is statistically inferior to deliberate training misfit: a shared high-variance signal is represented in the training span only together with orthogonal nuisance directions. The useful predictor must overshoot training labels to calibrate the shared signal, while the interpolating predictor under-calibrates it; the gap between the best-span and interpolation thresholds is a factor of order \(\gamma\) in sample size. The most transferable idea is an optimizer or training schedule that permits controlled positive training error instead of treating interpolation as the endpoint, implemented through a large constant-step one-pass SGD phase and checkpointing against validation performance. The construction is especially useful as a falsifiable intervention for overparameterized models with strong shared features and many example-specific directions, although its exact guarantees are limited to the stylized linear setting.