Exact Risk Ratios for Weighted Data Selection in Linear Regression

What the math gives to ML

The paper derives an explicit minimax excess-risk penalty for selecting only d+k weighted examples in d-dimensional least squares, with k between 1 and d-1. Its transferable contribution is a budget-sensitive geometric principle: whitened loss gradients can be decomposed into approximately orthogonal blocks, and balanced block ranks determine the unavoidable error from an undersized coreset. A practical neural-network adaptation is a gradient coreset for a linear probe, final classifier, or local linearized model, using whitening, block allocation, and nonnegative weights. The theorem is exact only for linear regression and orthogonal circuit-block configurations, so neural-network experiments should treat the Gamma quantity as a selection score and a falsifiable risk predictor rather than a universal guarantee.