{
 "artifacts": null,
 "category": "dynamics",
 "description": "Replace unconstrained momentum updates for selected neural-network parameter blocks by damped second-order motion on a unit sphere. The update learns directions while preserving the block norm exactly, and includes the centripetal term required for a valid acceleration constrained to the sphere rather than merely projecting gradients after an unconstrained step.",
 "formulas_latex": [
  "$$\\mathcal{M}=\\{\\xi\\in\\mathcal{H}:\\|\\xi\\|_{\\mathcal{H}}=1\\},\\qquad T_\\xi\\mathcal{M}=\\{\\eta\\in\\mathcal{H}:(\\eta,\\xi)_\\mathcal{H}=0\\}.$$",
  "$$\\ddot\\theta+\\gamma\\dot\\theta+g(\\theta)=\\lambda\\theta,\\qquad \\lambda=\\theta^\\top g(\\theta)+\\|\\dot\\theta\\|^2,$$",
  "$$\\ddot\\theta=-\\bigl(I-\\theta\\theta^\\top\\bigr)g(\\theta)-\\gamma\\dot\\theta-\\|\\dot\\theta\\|^2\\theta,$$",
  "$$\\frac{d}{dt}\\left(\\frac12\\|\\dot\\theta\\|^2+L(\\theta)\\right)=-\\gamma\\|\\dot\\theta\\|^2.$$"
 ],
 "id": 3087,
 "implementation": "(1) Integration point: apply this optimizer to one or more weight tensors, such as every Transformer attention projection matrix, each convolutional filter, or embedding row. Flatten a chosen block into $\\theta\\in\\mathbb{R}^n$ and initialize or rescale it to a fixed radius $r$; keep a separate velocity tensor $v$ of the same shape. Do not constrain biases initially. The forward pass uses the original tensor $\\theta$, so the architectural change is limited to optimizer state and update.\n\n(2) Pseudocode:\n```text\ninitialize theta with ||theta|| = r\nv = 0\nfor each minibatch:\n    loss = model(theta, batch)\n    g = autograd(loss, theta)\n    u = theta / r\n    Pgrad = g - u * dot(u, g)\n    a = -Pgrad - gamma * v - u * dot(v, v)\n    v_half = v + h * a\n    theta_trial = theta + h * v_half\n    theta_new = r * theta_trial / ||theta_trial||\n    u_new = theta_new / r\n    v_new = v_half - u_new * dot(u_new, v_half)\n    theta, v = theta_new, v_new\n```\nHere `Pgrad` is the tangent projection $P_u g$, the term `-u*dot(v,v)` is the centripetal correction from $\\langle u,\\ddot u\\rangle=-\\|v\\|^2$, and the final projection enforces tangency after discretization. Use $h$ as the optimizer step size and $\\gamma$ as damping; optionally use $v\\leftarrow\\beta v$ with $\\beta=\\exp(-\\gamma h)$ for an exact damping substep.\n\n(3) Computed versus estimated: the tangent projection, centripetal term, and norm/tangency checks are computed exactly from current tensors. No Hessian or PDE discretization is needed. Monitor $|\\|\\theta\\|-r|$ and $|\\theta^\\top v|$; both should remain near floating-point precision after each step. The energy diagnostic is $E=\\frac12\\|v\\|^2+L(\\theta)$, although minibatch noise means it need not decrease every step. Tune $h$ and $\\gamma$ on a logarithmic grid because excessive velocity can cause discrete-time oscillations.\n\n(4) First cheap experiment: train a 4-layer MLP and a small ViT-Tiny on CIFAR-10, applying the method only to linear weight matrices. Compare SGD with momentum, AdamW, and AdamW followed by per-step weight normalization, using matched parameter counts and approximately matched optimizer-state memory. Run three seeds and record training loss versus optimizer steps and measured FLOPs, validation accuracy, gradient norm, norm drift, and cosine alignment $\\theta^\\top v$. The first success signal is equal-or-faster loss descent with substantially smaller norm drift and fewer exploding or oscillatory runs than momentum. A secondary signal is improved validation accuracy or robustness at matched training loss. Ablate the centripetal term and final tangent projection to verify that gains come from spherical dynamics rather than ordinary weight renormalization.",
 "math_summary": "The paper defines the unit sphere $\\mathcal{M}=\\{\\xi\\in\\mathcal{H}:\\|\\xi\\|_{\\mathcal{H}}=1\\}$ and its tangent space $T_\\xi\\mathcal{M}=\\{\\eta\\in\\mathcal{H}:(\\eta,\\xi)_\\mathcal{H}=0\\}$. For a state $\\theta(t)\\in\\mathcal{M}$, differentiating $\\|\\theta\\|^2=1$ gives $(\\theta,\\dot\\theta)=0$ and $(\\theta,\\ddot\\theta)=-\\|\\dot\\theta\\|^2$. Let $P_\\theta=I-\\theta\\theta^\\top$ be the orthogonal projection onto $T_\\theta\\mathcal{M}$, let $g(\\theta)=\\nabla_\\theta L(\\theta)$ be the Euclidean loss gradient, let $v=\\dot\\theta$, and let $\\gamma\u003e0$ be damping. The constrained damped-gradient dynamics are $\\ddot\\theta+\\gamma\\dot\\theta+g(\\theta)=\\lambda(t)\\theta$, where the constraint gives $\\lambda=\\theta^\\top g(\\theta)+\\|v\\|^2$. Equivalently, $\\ddot\\theta=-P_\\theta g(\\theta)-\\gamma v-\\|v\\|^2\\theta$. If $L$ is restricted to the sphere and $E(t)=\\frac12\\|v\\|^2+L(\\theta)$, then $\\frac{dE}{dt}=-\\gamma\\|v\\|^2\\le0$. The neural adaptation applies this construction independently to a parameter block normalized to radius $r$: use $u=\\theta/r$, $P_u=I-uu^\\top$, and rescale the force consistently by $r$.",
 "math_tags": [
  "dynamical-systems",
  "geometry",
  "optimization",
  "control-theory"
 ],
 "ml_areas": [
  "optimizer",
  "training-dynamics",
  "attention"
 ],
 "paper": {
  "arxiv_id": "2609.02842",
  "arxiv_url": "https://arxiv.org/abs/2609.02842",
  "summary_what_math_gives_to_ml": "The paper's transferable core is a globally invariant second-order dynamical system on a Hilbert-sphere constraint: velocity remains tangent, while acceleration includes both a tangent force and the radial centripetal correction required by the unit-norm constraint. This gives a principled alternative to unconstrained momentum or repeated ad hoc weight normalization, while damping supplies an energy-dissipation mechanism. In neural networks, the construction can be used as a norm-preserving optimizer for parameter vectors, filters, embeddings, or attention heads, with the sphere radius fixing scale and the tangent projection controlling directional learning. The strongest initial test is a constrained momentum optimizer whose discrete update explicitly preserves both parameter norm and tangential velocity, compared against AdamW, SGD with momentum, and post-step weight normalization.",
  "title": "Global Well-posedness and Asymptotic Analysis of a Damped Nonlinear Wave Equation with a Codimension-One Constraint",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 6,
  "usefulness": 6
 },
 "solves": [
  "stability",
  "generalization",
  "accuracy"
 ],
 "title": "Damped Spherical Momentum",
 "url": "https://synthcore.org/idea/3087/damped-spherical-momentum",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}