{
 "artifacts": null,
 "category": "optimization",
 "description": "Replace an unconstrained adaptive optimizer step with a coordinate-wise trust-region step whose radius is determined by the current gradient and accumulated AdaGrad weight, then damp it using a curvature estimate and momentum. This can prevent unstable updates on sparse or highly anisotropic GNN parameters while retaining aggressive movement on coordinates with reliable gradients.",
 "formulas_latex": [
  "$$\\bm w_{k,i}=\\sqrt{\\sum_{t=1}^{k}g_{t,i}^{2}}+\\varepsilon,\\qquad \\Delta_{k,i}=\\frac{|g_{k,i}|}{w_{k,i}}.$$",
  "$$\\alpha_k=\\begin{cases}\\min\\left(1,\\dfrac{-\\langle\\bm g_k,\\bm s_k^S\\rangle}{\\langle\\bm s_k^S,\\bm B_k\\bm s_k^S\\rangle}\\right),\u0026\\text{if }\\langle\\bm s_k^S,\\bm B_k\\bm s_k^S\\rangle\u003e0,\\\\1,\u0026\\text{otherwise},\\end{cases}$$",
  "$$\\bm m_k=\\beta\\bm m_{k-1}+(1-\\beta)\\bm s_k^Q,\\qquad \\bm s_k^Q=\\alpha_k\\bm s_k^S,$$",
  "$$m_{k,i}\\leftarrow\\min\\bigl(\\max(m_{k,i},-\\Delta_{k,i}),\\Delta_{k,i}\\bigr),\\qquad \\bm\\theta_k=\\bm\\theta_{k-1}+\\bm m_k.$$"
 ],
 "id": 67,
 "implementation": "(1) Integration point: implement this as a drop-in replacement for the parameter-update portion of SGD or AdaGrad in a GNN, using the same gradient tensors and parameter groups. It can be applied after every local or coarse gradient computation from the two-level method, or tested independently on a full-graph baseline.\n\n(2) Pseudocode:\n```text\ninput theta, m = 0, w = 0\nfor k = 1,...,K:\n    g = gradient(loss(theta))\n    w = sqrt(w*w + g*g) + eps\n    sS = -eta * g / w\n    B_s = diagonal_curvature(sS)\n    q = dot(sS, B_s)\n    if q \u003e 0: alpha = min(1, -dot(g,sS)/q)\n    else:     alpha = 1\n    sQ = alpha * sS\n    m = beta*m + (1-beta)*sQ\n    m = clamp(m, -abs(g)/w, abs(g)/w)\n    theta = theta + m\n```\n(3) The accumulator, trust radius, curvature ratio, momentum, and clipping are computed from the displayed equations. For an inexpensive curvature model, use \\(B_{k,i}=g_{k,i}^2+\\lambda\\) as a diagonal Fisher proxy, or periodically estimate \\(\\langle s_k^S,B_ks_k^S\\rangle\\) with one Hessian-vector product. The mathematical ratio is exact for the chosen proxy; its usefulness must be measured empirically. (4) First cheap experiment: compare this optimizer against AdamW, AdaGrad, and the unmodified AG2m-style update on a 2-layer GCN and GraphSAGE using ogbn-arxiv or Cora. Keep batch size and gradient evaluations fixed. Plot training loss, validation accuracy, gradient norm, curvature ratio, fraction of clipped coordinates, and wall-clock time. Success means fewer divergence spikes, faster loss decrease at equal FLOPs, and equal-or-better validation accuracy. Ablate curvature correction, momentum, and clipping separately.",
 "math_summary": "At iteration \\(k\\), \\(\\bm g_k\\) is the current or subsampled gradient, \\(\\bm w_k\\) is the accumulated coordinate-wise AdaGrad weight, and the trust-region radius for coordinate \\(i\\) is \\(\\Delta_{k,i}=|g_{k,i}|/w_{k,i}\\). Let \\(\\bm s_k^S\\) be a proposed search step and \\(\\bm B_k\\) a symmetric curvature or Hessian approximation. The paper's scalar correction is \\(\\alpha_k=\\min(1,-\\langle\\bm g_k,\\bm s_k^S\\rangle/\\langle\\bm s_k^S,\\bm B_k\\bm s_k^S\\rangle)\\) when the denominator is positive, and \\(\\alpha_k=1\\) otherwise. A momentum state uses \\(\\bm m_k=\\beta\\bm m_{k-1}+(1-\\beta)\\bm s_k^Q\\), after which every coordinate is clipped to \\([-\\Delta_{k,i},\\Delta_{k,i}]\\). Here \\(\\bm s_k^Q=\\alpha_k\\bm s_k^S\\), \\(\\beta\\in[0,1)\\), and \\(\\bm s_k^S\\) can be a preconditioned negative-gradient step. The denominator measures predicted quadratic curvature; clipping enforces a coordinate trust region.",
 "math_tags": [
  "optimization",
  "numerical-analysis",
  "linear-algebra"
 ],
 "ml_areas": [
  "graph-nn",
  "optimizer",
  "training-dynamics"
 ],
 "paper": {
  "arxiv_id": "2608.22575",
  "arxiv_url": "https://arxiv.org/abs/2608.22575",
  "summary_what_math_gives_to_ml": "The paper combines domain decomposition with an AdaGrad-style trust-region optimizer to separate cheap local graph computation from occasional global information. The transferable asset is the two-level correction pattern: optimize on induced subgraphs or sampled local views, then restore long-range consistency using a much smaller coarse graph. Its coordinate-wise radius, curvature-scaled step acceptance, and momentum provide a concrete optimizer rather than merely a graph partitioning heuristic. The most promising ML use is scalable training of large GNNs, where local message passing dominates both memory and communication, with a secondary opportunity to use the same local/coarse schedule for other structured models.",
  "title": "Two-level domain-decomposition AdaGrad method for scalable training of graph neural networks",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 6,
  "usefulness": 7
 },
 "solves": [
  "stability",
  "speedup",
  "accuracy"
 ],
 "title": "Curvature-scaled coordinate trust-region momentum",
 "url": "https://synthcore.org/idea/67/curvature-scaled-coordinate-trust-region-momentum",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}