RPLSS: A randomized projected linear systems solver

What the math gives to ML

The paper's transferable asset is a projected randomized coordinate solver that stores previously discovered search directions and removes their contribution before accepting a new direction. Unlike ordinary randomized Kaczmarz or coordinate descent, the update is globally corrected by an explicit Gram-matrix projection, which can prevent repeated work in already-explained subspaces and can terminate after a finite number of independent directions. The most plausible neural-network use is not generic nonlinear training, but solving least-squares Gauss–Newton subproblems for a linear output head, LoRA adapter, or small last-layer block using minibatch Jacobian columns and partial parameter access.