{
 "artifacts": [
  {
   "name": "psd_lowrank_experiment.py",
   "url": "https://synthcore.org/code/1208/psd_lowrank_experiment.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1208/report.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/1208/results.json"
  }
 ],
 "category": "optimization",
 "description": "Approximate the minibatch loss Hessian by a positive-semidefinite bulk curvature plus a small signed transverse correction, and treat only the correction with explicit negative-curvature steps. This imports the paper's observation that all unstable directions can be confined to a low-dimensional subspace, producing a curvature-aware optimizer whose step-size boundary is governed by a small matrix rather than the full Hessian.",
 "download_zip": "https://synthcore.org/download/1208",
 "formulas_latex": [
  "$$H_{\\mathcal C}=B^{T}(D_{\\mathcal P}^{2}F)B=L^{\\Delta}+\\widetilde L,\\qquad L^{\\Delta}\\succeq0,\\qquad \\operatorname{rank}(\\widetilde L)\\le 2.$$",
  "$$\\widehat H=P+UCU^{T},\\qquad C=U^{T}(\\widehat H-P)U,\\qquad U^{T}U=I_r.$$",
  "$$\\eta_c=\\frac{2}{\\lambda_{\\max}\\!\\left(P^{-1/2}\\widehat H P^{-1/2}\\right)}.$$",
  "$$\\Delta\\theta=-\\eta P^{-1}g-\\eta U\\,\\phi(C,U^{T}P^{-1}g),\\qquad \\phi(C,z)=\\operatorname{diag}\\!\\left(\\frac{1}{1+\\gamma\\min(\\lambda_i(C),0)}\\right)z.$$"
 ],
 "id": 3025,
 "implementation": "Integrate this into AdamW or SGD immediately before the parameter update. Maintain a cheap diagonal or block-diagonal positive-semidefinite curvature estimate $P$ using exponential moving averages of squared gradients, Hutchinson diagonal Hessian estimates, or Gauss-Newton blocks. Every $K=20$ to $100$ steps, estimate the bottom $r=2$ or $4$ eigenvectors of the residual Hessian $H-P$ with two or three Lanczos iterations using Hessian-vector products; orthonormalize them to form $U$, and compute the small matrix $C$ from residual Hessian-vector products. Pseudocode: compute $g$ and $P^{-1}g$; periodically run Lanczos on $v\\mapsto Hv-Pv$; set $C_{ij}=u_i^T(Hu_j-Pu_j)$; diagonalize $C$; estimate the largest eigenvalue of $M=P^{-1/2}\\widehat H P^{-1/2}$; clip the global step to $0.9(2/\\lambda_{\\max}(M))$; optionally damp only eigenvectors associated with negative eigenvalues of $C$. The paper supplies the positive-semidefinite-plus-signed decomposition and low-rank instability principle; $P$, $U$, and $C$ are estimated empirically in the network. First experiment: train a 4-layer MLP on MNIST with SGD, AdamW, and this optimizer over a fixed learning-rate sweep. Log the first divergence boundary and the number of negative residual eigenvalues. The quantitative prediction is that the observed largest stable learning rate will be within 20% of $2/\\lambda_{\\max}(M)$, and that the residual negative spectrum will be concentrated in at most $r$ directions. Increasing $r$ from $2$ to $8$ should provide little additional stability if the low-rank mechanism transfers.",
 "math_summary": "The paper writes the constrained Hessian as $H_{\\mathcal C}=B^{T}(D_{\\mathcal P}^{2}F)B=L^{\\Delta}+\\widetilde L$, where $B$ embeds realizable pair coordinates into ambient pair space, $L^{\\Delta}\\succeq0$ is the gap Laplacian, and $\\widetilde L$ is the signed transverse Laplacian. For planar four-body configurations, $\\operatorname{rank}(\\widetilde L)\\le 2$, so negative curvature is determined by at most two generalized eigenvalues of a small symmetric pencil. In a neural network, let $g=\\nabla_\\theta \\ell(\\theta)$, let $P\\succeq0$ be a diagonal, block-diagonal, or low-rank-plus-diagonal estimate of the positive bulk, and let $U\\in\\mathbb R^{d\\times r}$ contain $r$ estimated residual curvature directions with $r\\ll d$. Use $\\widehat H=P+UCU^{T}$, where $C=U^{T}(\\widehat H-P)U$ is an $r\\times r$ symmetric signed matrix. For a locally positive quadratic model, the preconditioned update is stable when $\\eta\u003c2/\\lambda_{\\max}(P^{-1/2}\\widehat H P^{-1/2})$.",
 "math_tags": [
  "optimization",
  "linear-algebra",
  "spectral-theory",
  "dynamical-systems"
 ],
 "ml_areas": [
  "optimizer",
  "training-dynamics",
  "regularization"
 ],
 "paper": {
  "arxiv_id": "2609.01857",
  "arxiv_url": "https://arxiv.org/abs/2609.01857",
  "summary_what_math_gives_to_ml": "The paper provides a constructive Hessian decomposition in pair space: the constrained Hessian is a congruence-restricted ambient Hessian, and it splits into a positive-semidefinite gap Laplacian plus a signed transverse Laplacian containing all negative curvature. For non-collinear planar four-body configurations, the signed component has rank two, yielding a sharp low-dimensional description of instability and Morse index. Reflection symmetry further reduces the generalized eigenproblem into parity sectors, including a non-orthogonal reduction when the symmetry is not Euclidean-orthogonal. These mechanisms transfer well to neural-network optimization as low-rank negative-curvature tracking and symmetry-adapted Hessian/preconditioner blocks with measurable instability thresholds.",
  "title": "The Hessian of Planar Central Configurations in Pair Space: Decomposition, Morse Index and Symmetry Reduction",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 5,
  "usefulness": 8
 },
 "solves": [
  "stability",
  "speedup"
 ],
 "title": "PSD-plus-low-rank curvature optimizer",
 "url": "https://synthcore.org/idea/3025/psd-plus-low-rank-curvature-optimizer",
 "verification": {
  "peer_reviewed": false,
  "stage1_mechanism_check": {
   "worked": false,
   "confidence": 9,
   "verdict": "Built a PSD-plus-low-rank curvature optimizer with EMA diagonal PSD curvature, minibatch Hessian-vector products, rank-2 residual estimation, signed correction, and step clipping. The quadratic sanity check reproduced the predicted stability boundary: spectral radius was 0.90 at 0.95ηc and 1.10 at 1.05ηc, with one negative residual eigenvalue while the full Hessian remained PSD. On the fixed-seed digits MLP, however, the proposed method underperformed SGD substantially, so no neural-training win was observed.",
   "metrics": {
    "baseline": "SGD, lr=0.12, 90 steps: final loss 1.5128, accuracy 56.22%, diverged=false",
    "idea": "PSD-low-rank, nominal lr=0.12 with internal clipping, rank=2 and refresh every 15 steps: final loss 2.9691, minimum loss 1.6280, accuracy 15.78%, diverged=false",
    "toy": "ηc=1.4569; spectral radius 0.90 below the boundary and 1.10 above it; one negative residual eigenvalue"
   },
   "how_to_run": "python3 psd_lowrank_experiment.py",
   "files": [
    "psd_lowrank_experiment.py",
    "results.json"
   ],
   "limitations": "Only one small digits dataset, one nominal learning rate, one rank, one seed, and 90 update steps were tested; no learning-rate sweep or FLOP-normalized comparison was performed. The residual curvature was estimated with tiny randomized subspace iteration on minibatch Hessian-vector products, and the neural method was not tuned against SGD."
  },
  "status": "mechanism_failed",
  "status_label": "Mechanism failed",
  "updated_at": "2026-09-03T11:41:26",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": false,
    "tested": true
   }
  }
 }
}