{
 "artifacts": null,
 "category": "optimization",
 "description": "Replace the momentum update in a gradient optimizer by inertial motion plus a gradient-difference term, which discretely approximates Hessian-driven damping. Choose the damping coefficient and step size using the paper's refined stability inequality instead of the older restrictive bound, and adapt them whenever the estimated smoothness changes.",
 "formulas_latex": [
  "$$\\left(\\beta L\\sqrt{s}-1\\right)^2\u003c1+sL(1-sL),\\qquad 0\u003cs\\leq \\frac{1}{L},$$",
  "$$0\u003c\\beta L\\sqrt{s}\u003c1+\\sqrt{1+sL(1-sL)},$$",
  "$$x_{k+1}=x_k+\\left(1-\\frac{\\alpha}{k}\\right)(x_k-x_{k-1})-s\\nabla f(x_k)-\\beta\\bigl(\\nabla f(x_k)-\\nabla f(x_{k-1})\\bigr),$$",
  "$$L_{\\mathrm{est}}\\leftarrow\\max\\left(L_{\\mathrm{est}},\\frac{\\|g_k-g_{k-1}\\|_2}{\\|x_k-x_{k-1}\\|_2+\\varepsilon}\\right),\\qquad \\beta_{\\max}=\\frac{1+\\sqrt{1+sL_{\\mathrm{est}}(1-sL_{\\mathrm{est}})}}{L_{\\mathrm{est}}\\sqrt{s}}.$$"
 ],
 "id": 2932,
 "implementation": "Integrate this at the optimizer update for each parameter tensor, not inside the network forward pass. Maintain x_prev, x, g_prev, and g; use α=3 initially, s as the gradient step size, and β as the gradient-difference damping coefficient. At step k, compute g=grad(loss,x), estimate local smoothness from the secant ratio r=||g-g_prev||/(||x-x_prev||+ε), update L_est=max(decayed_L_est,r), and clip L_est to a robust percentile or exponential moving maximum to avoid one-batch outliers. Set s=min(s, 0.99/L_est). Compute bmax=[1+sqrt(1+s*L_est*(1-s*L_est))]/(L_est*sqrt(s)); choose β=min(β_target, 0.9*bmax), or use β=0.5*0.9*bmax for a less aggressive first run. Then execute x_next=x+(1-α/k)*(x-x_prev)-s*g-β*(g-g_prev), and shift x_prev=x and g_prev=g. At k=1, initialize x_prev=x and g_prev=g, so the damping term is zero. Estimate the mathematical quantities L_est and the admissible boundary empirically; do not claim a global L unless the model has a certified smoothness bound. First test on a 2-layer MLP and a small Transformer on MNIST or CIFAR-10, comparing AdamW, Nesterov momentum, fixed-β IGAHD, and the adaptive controller at equal optimizer steps, parameters, and measured FLOPs. Pre-register three mechanism tests: (1) on convex quadratics with known largest Hessian eigenvalue L, divergence should begin near βL√s=1+sqrt(1+sL(1-sL)), with boundary error below 10%; (2) at equal stable step size, the proposed method should reduce loss faster than β=0 in the intermediate damping range, measured by area under the training-loss curve; (3) reducing s while holding β fixed should remain stable once the refined condition holds, whereas the old β\u003c2√s rule would reject those settings. Remove only the β(g-g_prev) term as the ablation. The transfer is falsified if the adaptive method has no stability boundary near the predicted inequality on quadratics, or if gradient-difference damping consistently worsens equal-budget convergence after accounting for its extra gradient-memory cost.",
 "math_summary": "The paper considers a convex function f: R^d -\u003e R with L-Lipschitz gradient, meaning ||∇f(x)-∇f(y)|| \u003c= L||x-y||, a step size s satisfying 0 \u003c s \u003c= 1/L, and a Hessian-driven damping parameter β \u003e 0. Retaining two exact coefficients in the Lyapunov dissipation makes the relevant normalized symmetric 2x2 matrices converge to a positive-definite limit when (β L sqrt(s)-1)^2 \u003c 1+sL(1-sL). Equivalently, the sufficient admissible region is 0 \u003c β L sqrt(s) \u003c 1 + sqrt(1+sL(1-sL)); the simpler conservative condition β \u003c 2/(L sqrt(s)) is also sufficient. Use the gradient-difference approximation ∇f(x_k)-∇f(x_{k-1}) ≈ ∇²f(x_k)(x_k-x_{k-1}) as a discrete Hessian-damping force. The transfer hypothesis is that staying inside this region prevents oscillatory inertial divergence and permits a larger useful damping range than the old condition β \u003c 2 sqrt(s).",
 "math_tags": [
  "optimization",
  "convex-analysis",
  "dynamical-systems",
  "spectral-theory"
 ],
 "ml_areas": [
  "optimizer",
  "training-dynamics",
  "scheduler"
 ],
 "paper": {
  "arxiv_id": "2608.28088",
  "arxiv_url": "https://arxiv.org/abs/2608.28088",
  "summary_what_math_gives_to_ml": "The paper gives a sharper Lyapunov-stability region for inertial gradient descent with Hessian-driven damping by retaining exact coefficients in a positive-definiteness argument. The transferable asset is a computable coupling between step size, smoothness estimate, and gradient-difference damping strength, which permits damping settings excluded by the older condition and remains valid for sufficiently small steps. A practical adaptation is an IGAHD optimizer with online smoothness estimation and a parameter controller that enforces the refined inequality while maximizing the admissible step size. The quadratic spectral analysis also supplies a useful diagnostic: on nearly quadratic problems, stable training should exhibit geometric decay rather than merely nonincreasing loss.",
  "title": "A Refined Parameter Condition in the Lyapunov Analysis of IGAHD",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 5,
  "usefulness": 6
 },
 "solves": [
  "speedup",
  "stability",
  "accuracy"
 ],
 "title": "Lyapunov-certified Hessian-damped optimizer",
 "url": "https://synthcore.org/idea/2932/lyapunov-certified-hessian-damped-optimizer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}