{
 "artifacts": null,
 "category": "optimization",
 "description": "Replace the usual Euclidean gradient step by a warped resolvent step with a learned diagonal metric, and explicitly account for the fact that minibatch or approximate gradients are non-monotone. The metric and step scale are increased when an empirical semimonotonicity test detects negative curvature or gradient mismatch, producing a practical stability safeguard without requiring the neural loss itself to be convex.",
 "formulas_latex": [
  "$$\\text{find }x\\in\\mathcal{H}\\text{ such that }0\\in(A+L^{*}BL+C+D)x.$$",
  "$$z=(\\bm M+\\bm A)^{-1}(\\bm Mx-Fx),\\qquad F=C+D.$$",
  "$$\\hat{\\rho}=\\min\\{\\rho,0\\},\\qquad \\underline{\\tau}_{\\zeta}=-\\frac{\\zeta\\hat{\\rho}}{\\beta+\\hat{\\rho}},\\qquad \\tau\u003e\\underline{\\tau}_{\\zeta}.$$",
  "$$\\langle Fx-Fy,x-y\\rangle\\geq\\rho\\|x-y\\|^{2},\\qquad \\|Dx-Dy\\|\\leq\\vartheta\\|x-y\\|.$$"
 ],
 "id": 71,
 "implementation": "(1) Integration point: replace the AdamW or SGD parameter update during pretraining or fine-tuning. Partition the minibatch gradient into a stable component $c_t$ and residual component $d_t$; the simplest MVP uses $c_t=0$ and treats the full gradient as $d_t$, while a stronger version uses a moving-average gradient as $c_t$ and the minibatch-minus-moving-average residual as $d_t$. Store a positive diagonal metric $M_t$ (one scalar per parameter tensor or one EMA second moment per parameter). Set $A=\\partial R$, where $R$ is weight decay or a proximal constraint. (2) Pseudocode: compute $g_t=C_t+D_t$; draw or retain the previous minibatch gradient $g_{t-1}$; estimate $\\rho_t=\\langle g_t-g_{t-1},w_t-w_{t-1}\\rangle/(\\|w_t-w_{t-1}\\|^2+\\epsilon)$ and use $\\hat\\rho_t=\\min(\\rho_t,0)$. Estimate $\\beta_t$ from the stable component with $\\beta_t=\\|c_t-c_{t-1}\\|^2/(\\langle c_t-c_{t-1},w_t-w_{t-1}\\rangle+\\epsilon)$, clipped to a safe interval. Choose $\\zeta=0.25$, $\\tau_t=1.1[-\\zeta\\hat\\rho_t/(\\beta_t+\\hat\\rho_t)]_+$, plus a fixed positive floor. Form $v_t=w_t-M_t^{-1}g_t/\\tau_t$ and apply the metric proximal map $w_{t+1}=\\operatorname{prox}^{M_t}_{R}(v_t)$, where $\\operatorname{prox}^{M}_{R}(v)=\\arg\\min_w R(w)+\\frac12\\|w-v\\|_M^2$. Backtrack by multiplying $\\tau_t$ by two if the observed secant inequality violates the chosen bound. (3) The paper supplies the warped-resolvent structure and explicit dependence on negative semimonotonicity; the secant estimates of $\\rho_t$, $\\beta_t$, and the diagonal metric are empirical engineering approximations. (4) First experiment: train a 6-layer MLP or small ViT on CIFAR-10, comparing the method with AdamW at equal parameter count, minibatch size, and FLOPs. Measure loss decrease per optimizer step, gradient-norm spikes, divergence frequency over five seeds, and final validation accuracy. The first success signal is fewer exploding-loss runs and faster loss descent at the same compute; a secondary signal is improved accuracy under deliberately noisy or stale gradients.",
 "math_summary": "The paper studies inclusions of the form $0\\in(A+C+D)x+L^{*}u$ and, in the primal-only case, $0\\in(A+L^{*}BL+C+D)x$, where $A$ is a set-valued proximal operator, $C$ is $\\beta$-cocoercive, and $D$ is $\\vartheta$-Lipschitz but may be non-monotone. Cocoercivity means $\\langle Cx-Cy,x-y\\rangle\\geq\\beta\\|Cx-Cy\\|^{2}$ for $\\beta\u003e0$; Lipschitzness means $\\|Dx-Dy\\|\\leq\\vartheta\\|x-y\\|$. The warped resolvent uses an invertible operator $\\bm M$ and the update $z=(\\bm M+\\bm A)^{-1}(\\bm Mx-Fx)$, with $F=C+D$. The extracted convergence assumption introduces a nonexpansive map $\\bm S$, a scale $\\tau$, and $\\zeta\\in[0,1/2)$ such that $\\tau\\bm M-\\bm S$ is $\\zeta$-Lipschitz relative to $\\bm S$. Its explicit admissible lower bound is $\\tau\u003e\\underline{\\tau}_{\\zeta}$, where $\\hat\\rho=\\min\\{\\rho,0\\}$ and $\\underline{\\tau}_{\\zeta}=-\\zeta\\hat\\rho/(\\beta+\\hat\\rho)$. Here $\\rho$ is the semimonotonicity modulus, meaning $\\langle Fx-Fy,x-y\\rangle\\geq\\rho\\|x-y\\|^{2}$; negative $\\rho$ quantifies non-monotonicity. For optimization, take $A=\\partial R$ for a regularizer $R$, $C$ as a stable or cocoercive gradient component, $D$ as the non-monotone residual, $S=I$, and $M$ as a positive diagonal preconditioner.",
 "math_tags": [
  "optimization",
  "convex-analysis",
  "dynamical-systems",
  "numerical-analysis"
 ],
 "ml_areas": [
  "optimizer",
  "training-dynamics"
 ],
 "paper": {
  "arxiv_id": "2608.22687",
  "arxiv_url": "https://arxiv.org/abs/2608.22687",
  "summary_what_math_gives_to_ml": "The paper develops a forward-backward splitting framework that remains convergent when one forward operator is Lipschitz but non-monotone, a regime that includes approximate adjoints and imperfect gradient-like operators. The transferable asset is the warped resolvent: a variable metric or nonlinear preconditioner can absorb part of the operator mismatch, while an explicit semimonotonicity-dependent lower bound controls instability. A promising neural-network adaptation is a safeguarded proximal optimizer for nonconvex training, using curvature or minibatch-gradient mismatch estimates to choose the metric scale and step size rather than assuming ordinary monotonicity of the loss gradient.",
  "title": "Nonlinear Forward-Backward Algorithm for Solving Non-monotone+Lipschitz Inclusions with Applications to Adjoint Mismatch Problems",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 7,
  "usefulness": 7
 },
 "solves": [
  "stability",
  "speedup",
  "generalization"
 ],
 "title": "Warped nonmonotone proximal optimizer",
 "url": "https://synthcore.org/idea/71/warped-nonmonotone-proximal-optimizer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}