Sequential Preconditioned Conjugate Gradient Method for Linear Statistical Models

What the math gives to ML

The paper's transferable contribution is a continuation strategy for large least-squares systems: solve a sequence of increasingly accurate sketched problems, warm-start each solve from the previous solution, and reserve a short full-data refinement for the end. This can turn expensive repeated Jacobian or feature-matrix products into mostly cheap computations on small row subsets while retaining the accuracy of the full normal-equation solution. The most direct neural-network use is solving linearized Gauss-Newton or last-layer/LoRA least-squares subproblems, where the same large design matrix is repeatedly used and conjugate-gradient iterations are otherwise dominated by full-batch matrix products.