Barzilai-Borwein Fails Superlinear Convergence on an Open Set of Quadratics for Every Dimension $n\geq 4$

What the math gives to ML

The paper exposes a concrete failure mode of the BB1 optimizer: an attracting periodic orbit in the projectivized gradient dynamics can produce only geometric convergence, even on strictly convex quadratics and for an open set of initial conditions. This is valuable for neural optimization because BB-style scalar curvature adaptation is cheap but can create persistent oscillatory regimes that are difficult to detect from the loss alone. The transferable mechanism is a cycle detector operating on normalized gradients and BB step sizes, coupled to a fallback such as BB2 or safeguarded steepest descent. The resulting optimizer is directly testable on deterministic quadratics before evaluation on small neural networks.