NGMRES convergence analysis and proof of acceleration for contractive and noncontractive iterations

What the math gives to ML

The paper analyzes nonlinear GMRES as a residual-minimizing acceleration of arbitrary nonlinear fixed-point iterations, including noncontractive ones. Its transferable asset is not merely the mixing rule, which resembles Anderson acceleration, but the explicit use of least-squares residual reduction to predict whether a chosen history depth will improve the next iteration and to trigger depth changes or restarts. A practical neural-network use is to accelerate equilibrium-model inference or recurrent implicit layers by replacing plain Picard iterations with safeguarded NGMRES and selecting the history depth from the observed residual-ratio gain.