Numerical Study of Eigenvector Deflation to Accelerate the WaveHoltz Method

What the math gives to ML

The paper develops a concrete spectral acceleration mechanism: explicitly remove troublesome eigenmodes from a slowly converging iteration, then solve the remaining problem in the complementary subspace. The transferable asset is the combination of targeted eigenvector deflation, augmented Krylov correction, and low-rank storage of deflation vectors. A promising neural-network application is damped Newton, Gauss-Newton, or natural-gradient training, where small-curvature Hessian or Fisher directions make inner conjugate-gradient solves poorly conditioned. The first test should use Hessian-vector products to estimate a few low-curvature modes and compare deflated versus ordinary CG at equal HVP and wall-clock budgets.