Minimal Negative-Curvature L-BFGS

Integrate the method into the curvature-pair insertion step of a standard L-BFGS optimizer, rather than changing the network architecture. Use it for full-batch or large-batch fine-tuning, where gradients at consecutive iterates are sufficiently comparable. Maintain the usual memory of the most recent m pairs (s_i,y_i), and use the existing two-loop recursion to compute p_k=-H_k g_k. After a line search produces x_{k+1}, compute s=x_{k+1}-x_k and y=g_{k+1}-g_k. Let c=s^T y and use a scale-aware negative-curvature test c < -epsilon_curv ||s||_2 ||y||_2, with epsilon_curv initially 1e-4. If the test is false, insert the ordinary pair. If true, choose one variant: sign flip ytilde=-y; Euclidean projection ytilde=y+(abs(c)-c)s/(s^T s); or inverse-metric projection ytilde=y+(abs(c)-c)B s/(s^T B s). The inverse-metric version requires applying the current limited-memory B operator to s; implement this through the stored L-BFGS pairs, or initially test the Euclidean version. Enforce ctilde=max(abs(c),epsilon_sec ||s||_2 ||y||_2) if floating-point noise makes the denominator too small. Insert (s,ytilde), then run the unchanged two-loop recursion. The paper-derived quantities are the curvature test, projection formulas, and positive-secant guarantee; curvature is estimated empirically from consecutive gradients, and B actions are estimated from the current L-BFGS memory. First experiment: fine-tune a 10–50M parameter Transformer or ResNet on a small dataset using a finite minimax loss over four augmentations or task heads. Compare AdamW, ordinary L-BFGS, damped L-BFGS, and all three correction rules. Track loss versus gradient evaluations and wall-clock time, fraction of corrected pairs, line-search failure rate, NaNs, and final worst-case component loss. Success means fewer rejected line searches and faster decrease of the maximum component loss than ordinary L-BFGS without a significant per-step overhead.