{
 "artifacts": [
  {
   "name": "experiment.py",
   "url": "https://synthcore.org/code/26/experiment.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/26/report.md"
  }
 ],
 "category": "dynamics",
 "description": "Replace ordinary first-order updates of unit-normalized class prototypes or embedding vectors with inertial motion constrained to the sphere. The velocity is kept tangent to the sphere, gradients are projected into the tangent space, and a retraction returns the representation to unit norm after each step. This is a geometrically explicit alternative to applying momentum followed by ad hoc normalization.",
 "download_zip": "https://synthcore.org/download/26",
 "formulas_latex": [
  "$$m\\left(\\ddot{\\boldsymbol{\\sigma}}_{i}+\\left\\|\\dot{\\boldsymbol{\\sigma}}_{i}\\right\\|^{2}\\boldsymbol{\\sigma}_{i}\\right)+\\dot{\\boldsymbol{\\sigma}}_{i}=\\mathbf{W}_{i}\\boldsymbol{\\sigma}_{i}+\\frac{\\lambda}{N}\\sum_{j=1}^{N}\\left[\\boldsymbol{\\sigma}_{j}-\\left(\\boldsymbol{\\sigma}_{j}\\cdot\\boldsymbol{\\sigma}_{i}\\right)\\boldsymbol{\\sigma}_{i}\\right],\\qquad \\|\\boldsymbol{\\sigma}_i\\|=1.$$$$P_x=I-xx^\\top,\\qquad P_x\\nabla_xL=\\nabla_xL-(x^\\top\\nabla_xL)x.$$$$v_i\\leftarrow\\beta v_i-\\eta P_{x_i}\\nabla_{x_i}L,\\qquad x_i\\leftarrow\\frac{x_i+v_i}{\\|x_i+v_i\\|},\\qquad v_i\\leftarrow(I-x_ix_i^\\top)v_i.$$$$"
 ],
 "id": 100,
 "implementation": "(1) Integration point: apply this optimizer only to a tensor of unit-normalized vectors, initially class prototypes in a contrastive-learning head or token/feature prototypes in a retrieval layer. Keep the backbone optimizer unchanged for the first experiment. Let x have shape [N,D], with every row normalized, and let v have the same shape as a persistent optimizer state.\n\n(2) Pseudocode:\n```text\ninitialize x = normalize(x)\ninitialize v = zeros_like(x)\nfor each minibatch:\n    loss = network_loss(backbone(batch), x)\n    g = grad(loss, x)\n    gx = g - row_sum(g*x, axis=D, keepdim=True)*x\n    v = beta*v - eta*gx\n    x_trial = x + v\n    x = x_trial / (row_norm(x_trial) + eps)\n    v = v - row_sum(v*x, axis=D, keepdim=True)*x\n```\nUse beta=exp(-eta/m) as a damping-inspired default and compare beta values 0.9, 0.95, and 0.99. A semi-implicit variant can use v=(beta*v-eta*gx)/(1+eta/m), with m and damping exposed as hyperparameters.\n\n(3) Computed from the paper's mathematics: the tangent projector, unit-sphere constraint, inertial velocity, and retraction. Estimated empirically: eta, beta or m, gradient-noise sensitivity, and whether coupling across prototypes helps. Optionally add the paper's mean-field coupling as c_i=lambda*(mu-(mu dot x_i)*x_i), where mu=(1/N)sum_j x_j, but begin with lambda=0 to isolate the optimizer effect.\n\n(4) Cheap experiment: train a ResNet-18 or ViT-Tiny on CIFAR-100 with a supervised contrastive head containing 100 unit class prototypes. Compare SGD with momentum, AdamW plus post-update normalization, and this spherical inertial optimizer at matched parameter-update FLOPs. Record training loss versus optimizer steps, validation accuracy, prototype norm error, gradient-to-update ratio, and oscillation after learning-rate reductions. Success means faster loss descent or higher accuracy at equal steps without norm drift or prototype collapse; target at least 10% fewer steps to the same validation accuracy.",
 "math_summary": "The paper models each state as a unit vector sigma_i in R^D with ||sigma_i||=1. Its inertial equation is m(ddot(sigma_i)+||dot(sigma_i)||^2 sigma_i)+dot(sigma_i)=W_i sigma_i+(lambda/N) sum_j[sigma_j-(sigma_j dot sigma_i)sigma_i]. Here m is inertia, sigma_i is the normalized state, W_i is an antisymmetric rotation matrix, lambda is coupling strength, and N is the number of agents. The term ||dot(sigma_i)||^2 sigma_i is the curvature correction obtained by differentiating ||sigma_i||^2=1 twice. For a neural optimizer, set W_i=0, define the tangent projector P_x=I-xx^T, and replace the interaction force by the projected negative gradient -P_x grad_x L. A practical discrete approximation stores tangent velocity v_i, updates v_i with momentum and projected gradient, retracts x_i to unit norm, and removes any radial component from the velocity.",
 "math_tags": [
  "geometry",
  "dynamical-systems",
  "optimization",
  "linear-algebra"
 ],
 "ml_areas": [
  "optimizer",
  "embedding",
  "training-dynamics"
 ],
 "paper": {
  "arxiv_id": "2608.23426",
  "arxiv_url": "https://arxiv.org/abs/2608.23426",
  "summary_what_math_gives_to_ml": "The paper supplies a concrete second-order dynamical system on a unit sphere: the acceleration correction ||dot(sigma)||^2 sigma keeps each state on the manifold while inertia transports velocity along it. This can be transferred to neural modules whose representations or prototypes are constrained to unit norm, replacing first-order projected updates with geometrically valid inertial updates. The likely benefit is faster movement across flat regions and better collective responsiveness among many normalized embeddings, while the sphere constraint prevents norm drift. The first implementation should test the optimizer/module transfer independently of the paper's synchronization phase-transition claims.",
  "title": "Inertial synchronization of networked oscillators in arbitrary dimensions",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 5,
  "usefulness": 6
 },
 "solves": [
  "speedup",
  "stability",
  "accuracy"
 ],
 "title": "Riemannian inertial updates for normalized embeddings",
 "url": "https://synthcore.org/idea/100/riemannian-inertial-updates-for-normalized-embeddings",
 "verification": {
  "peer_reviewed": false,
  "stage1_mechanism_check": {
   "worked": false,
   "confidence": 9,
   "verdict": "Built a synthetic normalized-prototype classification benchmark comparing ambient momentum plus normalization against the proposed tangent-projected inertial update with retraction. Projector, retraction, and tangent-velocity checks passed at approximately 1e-7 numerical error. Across three seeds and beta values 0.9, 0.95, and 0.99, the idea showed no accuracy win; beta=0.99 modestly improved training loss but remained worse in final accuracy than baseline.",
   "metrics": {
    "baseline": "beta=0.9: final accuracy 0.42875, best accuracy 0.43833, final train loss 1.58014, norm error 1.79e-7; beta=0.99: final accuracy 0.40625, loss 1.71472",
    "idea": "beta=0.9: final accuracy 0.42833, best accuracy 0.43417, final train loss 1.58012, norm error 1.79e-7, tangent velocity error 1.82e-8; beta=0.99: accuracy 0.42250, loss 1.63279"
   },
   "how_to_run": "python3 experiment.py",
   "files": [
    "experiment.py"
   ],
   "limitations": "Only a small synthetic full-batch prototype classifier was tested; no CIFAR or MNIST backbone, contrastive objective, AdamW comparison, FLOP-matched timing, learning-rate schedules, coupling term, or large-scale generalization study was evaluated."
  },
  "status": "mechanism_failed",
  "status_label": "Mechanism failed",
  "updated_at": "2026-08-29T22:47:22",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": false,
    "tested": true
   }
  }
 }
}