{
 "artifacts": null,
 "category": "optimization",
 "description": "Modify an L-BFGS curvature pair only when the observed secant curvature is negative. Replace the gradient-difference vector by the smallest Euclidean or inverse-metric correction that enforces positive curvature, then use the unmodified BFGS update and two-loop recursion. This avoids the computational and conditioning cost of adding a large isotropic damping term to the whole inverse-Hessian approximation.",
 "formulas_latex": [
  "$$s_k=x_{k+1}-x_k,\\qquad y_k=g_{k+1}-g_k,\\qquad s_k^T y_k\u003c0,$$",
  "$$s_k^T\\widetilde y_k=|s_k^T y_k|,$$",
  "$$\\widetilde y_k^{\\mathrm{Euc}}=y_k+\\frac{|s_k^T y_k|-s_k^T y_k}{s_k^T s_k}s_k,$$",
  "$$\\widetilde y_k^{B^{-1}}=y_k+\\frac{|s_k^T y_k|-s_k^T y_k}{s_k^T B_k s_k}B_k s_k,\\qquad B_{k+1}=B_k-\\frac{B_k s_k s_k^T B_k}{s_k^T B_k s_k}+\\frac{\\widetilde y_k\\widetilde y_k^T}{s_k^T\\widetilde y_k}.$$"
 ],
 "id": 2822,
 "implementation": "Integrate the method into the curvature-pair insertion step of a standard L-BFGS optimizer, rather than changing the network architecture. Use it for full-batch or large-batch fine-tuning, where gradients at consecutive iterates are sufficiently comparable. Maintain the usual memory of the most recent m pairs (s_i,y_i), and use the existing two-loop recursion to compute p_k=-H_k g_k. After a line search produces x_{k+1}, compute s=x_{k+1}-x_k and y=g_{k+1}-g_k. Let c=s^T y and use a scale-aware negative-curvature test c \u003c -epsilon_curv ||s||_2 ||y||_2, with epsilon_curv initially 1e-4. If the test is false, insert the ordinary pair. If true, choose one variant: sign flip ytilde=-y; Euclidean projection ytilde=y+(abs(c)-c)s/(s^T s); or inverse-metric projection ytilde=y+(abs(c)-c)B s/(s^T B s). The inverse-metric version requires applying the current limited-memory B operator to s; implement this through the stored L-BFGS pairs, or initially test the Euclidean version. Enforce ctilde=max(abs(c),epsilon_sec ||s||_2 ||y||_2) if floating-point noise makes the denominator too small. Insert (s,ytilde), then run the unchanged two-loop recursion. The paper-derived quantities are the curvature test, projection formulas, and positive-secant guarantee; curvature is estimated empirically from consecutive gradients, and B actions are estimated from the current L-BFGS memory. First experiment: fine-tune a 10–50M parameter Transformer or ResNet on a small dataset using a finite minimax loss over four augmentations or task heads. Compare AdamW, ordinary L-BFGS, damped L-BFGS, and all three correction rules. Track loss versus gradient evaluations and wall-clock time, fraction of corrected pairs, line-search failure rate, NaNs, and final worst-case component loss. Success means fewer rejected line searches and faster decrease of the maximum component loss than ordinary L-BFGS without a significant per-step overhead.",
 "math_summary": "Let x_k be the parameter vector at iteration k, s_k=x_{k+1}-x_k the step, and y_k=g_{k+1}-g_k the gradient-difference vector, where g_k is the gradient of the current smooth objective. A pair has strict negative curvature when s_k^T y_k\u003c0. The correction target is s_k^T\\widetilde y_k=|s_k^T y_k|\u003e0. The sign-flip rule uses \\widetilde y_k=-y_k. The Euclidean nearest-point rule solves min_y ||y-y_k||_2^2 subject to s_k^T y=|s_k^T y_k|, giving \\widetilde y_k=y_k+(|s_k^T y_k|-s_k^T y_k)s_k/(s_k^T s_k). If B_k is the positive-definite L-BFGS Hessian approximation, the B_k^{-1}-metric rule solves min_y (y-y_k)^T B_k^{-1}(y-y_k) under the same secant constraint, giving \\widetilde y_k=y_k+(|s_k^T y_k|-s_k^T y_k)B_k s_k/(s_k^T B_k s_k). The standard BFGS update is then B_{k+1}=B_k-(B_k s_k s_k^T B_k)/(s_k^T B_k s_k)+(\\widetilde y_k\\widetilde y_k^T)/(s_k^T\\widetilde y_k). Because both denominators are positive, positive definiteness is preserved when B_k is positive definite.",
 "math_tags": [
  "optimization",
  "convex-analysis",
  "numerical-analysis"
 ],
 "ml_areas": [
  "optimizer",
  "loss",
  "training-dynamics",
  "fine-tuning"
 ],
 "paper": {
  "arxiv_id": "2608.29300",
  "arxiv_url": "https://arxiv.org/abs/2608.29300",
  "summary_what_math_gives_to_ml": "The paper provides a practical way to make limited-memory quasi-Newton training robust to locally indefinite curvature without eigendecompositions, Krylov searches, or costly isotropic damping. Its key transferable asset is a minimal correction of only those secant pairs whose curvature is genuinely negative, while retaining the standard positive-definite BFGS update and two-loop recursion. This is especially relevant for optimizing finite worst-case objectives such as maxima over tasks, augmentations, experts, or adversarial scenarios, where nonsmoothness and negative curvature are common. The most direct neural-network test is full-batch or large-batch L-BFGS fine-tuning with an adaptive negative-curvature pair correction.",
  "title": "Negative-Curvature-Informed L-BFGS via Minimal Secant Corrections for Finite Minimax Problems",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 7,
  "usefulness": 7
 },
 "solves": [
  "stability",
  "speedup",
  "accuracy"
 ],
 "title": "Minimal Negative-Curvature L-BFGS",
 "url": "https://synthcore.org/idea/2822/minimal-negative-curvature-l-bfgs",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}