The Sharp Worst-Case Asymptotic Rate of the Barzilai--Borwein Method in $\mathbb R^d$ and Hilbert Spaces

What the math gives to ML

The paper gives a sharp spectral explanation of the Barzilai--Borwein method: on strongly convex quadratics, the worst-case asymptotic gradient factor is exactly governed by the active condition number, c_H = (kappa(H)-1)/(kappa(H)+1). Its transferable asset is the delayed Rayleigh-quotient update, which estimates curvature from consecutive gradients without explicitly forming a Hessian. A practical neural-network adaptation is a safeguarded, blockwise BB optimizer with gradient smoothing and explicit protection against noisy-minibatch denominators.