Learning Asymptotics with Convergence-Rate Guarantees using Linear Least Squares

What the math gives to ML

The paper's transferable asset is a disciplined way to estimate unknown constants and convergence rates from a sequence that is known or hypothesized to have an asymptotic expansion. Sliding least squares and its Tikhonov-regularized variant provide online estimators using only recent observations, while the analysis highlights that window size, parity effects, and higher-order terms can cause slow convergence or divergence. In neural-network training, this suggests an asymptotic-aware controller for validation loss or parameter updates: estimate the limiting metric and remaining improvement, then stop or reallocate compute only when estimates are stable across windows. The method is most promising for expensive fine-tuning and hyperparameter sweeps, where reliable early termination can save compute without changing the trained model.