{
 "artifacts": [
  {
   "name": "README.md",
   "url": "https://synthcore.org/code/1229/README.md"
  },
  {
   "name": "bench_report.json",
   "url": "https://synthcore.org/code/1229/bench_report.json"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1229/report.md"
  },
  {
   "name": "report_bench_2026-09-03T131319.md",
   "url": "https://synthcore.org/code/1229/report_bench_2026-09-03T131319.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/1229/results.json"
  },
  {
   "name": "run_experiment.py",
   "url": "https://synthcore.org/code/1229/run_experiment.py"
  },
  {
   "name": "stage2_dynamics.py",
   "url": "https://synthcore.org/code/1229/stage2_dynamics.py"
  }
 ],
 "category": "dynamics",
 "description": "Add a fading-memory consensus force to parameter dynamics, pulling the current parameter toward a distributed average of its past while preserving the ordinary gradient step. Implement the infinite memory through one or several recursive exponential states, and tune the memory decay so that quadratic-mode dynamics remain exponentially stable. This should suppress oscillations and catastrophic steps without relying on conventional momentum alone.",
 "download_zip": "https://synthcore.org/download/1229",
 "formulas_latex": [
  "$$\\dot{x}(t)=-\\nabla L(x(t))+\\kappa\\int_0^\\infty\\mu(s)\\bigl[x(t-s)-x(t)\\bigr]ds,\\qquad \\mu(s)\\geq0,\\quad\\int_0^\\infty\\mu(s)ds=1.$$",
  "$$z(t,s)=x(t)-x(t-s),\\qquad \\partial_tz(t,s)+\\partial_sz(t,s)=\\dot{x}(t),\\qquad z(t,0)=0.$$",
  "$$V(t)=\\frac12\\lVert x(t)-x_*\\rVert^2+\\frac{\\kappa}{2}\\int_0^\\infty\\mu(s)\\lVert z(t,s)\\rVert^2ds,\\qquad \\mu'(s)\\leq-\\delta\\mu(s).$$",
  "$$m_t=\\beta(x-m),\\qquad \\dot{x}=-\\lambda x+\\kappa(m-x),\\qquad r^2+(\\lambda+\\kappa+\\beta)r+\\beta\\lambda=0.$$"
 ],
 "id": 3057,
 "implementation": "1. Integration point: apply this after the gradient is computed in SGD or AdamW, initially using one memory tensor per parameter block rather than one state per scalar. Maintain an exponential memory m of parameters and add a restoring term kappa times (m minus theta) to the update. The discrete equations are theta equals theta minus eta times g plus eta times kappa times (m minus theta), followed by m equals rho times m plus one minus rho times theta, with rho equal to exp of minus beta eta. 2. Pseudocode: initialize theta and m to theta_0; each step compute g equal to the gradient of L(theta); update theta using the gradient and memory force; update m using the exponential average; log both the gradient and memory-force norms. Optionally clip only the memory force to a fixed multiple of the gradient norm. 3. Computed from the mechanism: rho and the quadratic characteristic polynomial. Estimated empirically: local curvature lambda using Hessian-vector products or Rayleigh quotients, and the observed decay rate from parameter trajectories. 4. First cheap experiment: train a two-layer MLP on MNIST and a small Transformer on WikiText-2, comparing SGD, SGD with momentum, and this optimizer over matched learning-rate grids. First use a quadratic problem with known Hessian eigenvalues. The quantitative prediction is that measured modal decay rates follow the smaller-magnitude root of the polynomial, and the largest stable learning rate agrees with the discrete linearized prediction within 20 percent. The Lyapunov-like sum of parameter error and weighted memory discrepancy should decrease monotonically after transients.",
 "math_summary": "For a parameter mode x(t), use the Volterra dynamics \\(\\dot{x}(t)=-\\nabla L(x(t))+\\kappa\\int_0^\\infty\\mu(s)[x(t-s)-x(t)]ds\\), where \\(\\kappa\\geq0\\) is memory strength and \\(\\mu(s)\\geq0\\) is normalized by \\(\\int_0^\\infty\\mu(s)ds=1\\). Define the Dafermos history variable \\(z(t,s)=x(t)-x(t-s)\\), satisfying \\(\\partial_tz+\\partial_sz=\\dot{x}(t)\\) and \\(z(t,0)=0\\). For a quadratic loss mode with curvature \\(\\lambda\\geq0\\), a history energy combines parameter error and weighted history discrepancy. If \\(\\mu'(s)\\leq-\\delta\\mu(s)\\), the history transport term dissipates at rate at least \\(\\delta\\). For the exponential kernel \\(\\mu(s)=\\beta e^{-\\beta s}\\), the exact Markovian realization is \\(m_t=\\beta(x-m)\\), \\(\\dot{x}=-\\lambda x+\\kappa(m-x)\\), with characteristic polynomial \\(r^2+(\\lambda+\\kappa+\\beta)r+\\beta\\lambda=0\\).",
 "math_tags": [
  "dynamical-systems",
  "control-theory",
  "functional-analysis",
  "optimization"
 ],
 "ml_areas": [
  "optimizer",
  "training-dynamics",
  "scheduler"
 ],
 "paper": {
  "arxiv_id": "2609.02454",
  "arxiv_url": "https://arxiv.org/abs/2609.02454",
  "summary_what_math_gives_to_ml": "The paper's transferable mechanism is that infinite fading-memory interactions can be rewritten as a Markovian dynamical system on an augmented history space, where the memory itself supplies a dissipative Lyapunov term. Under a nonnegative decreasing kernel with exponential decay, disagreement decays exponentially; for flocking, a divergent-tail influence function yields unconditional alignment. A useful neural-network transfer is a history-augmented optimizer or recurrent state update in which parameters or hidden states are attracted to a fading average of their own past. The key falsifiable prediction is an exponential decay rate and stability boundary governed by the memory decay and local curvature or recurrent gain.",
  "title": "Exponential Consensus and Flocking in Multi-Agent Systems with Infinite Fading Memory",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 6,
  "usefulness": 7
 },
 "solves": [
  "stability",
  "speedup"
 ],
 "title": "Lyapunov Fading-Memory Optimizer",
 "url": "https://synthcore.org/idea/3057/lyapunov-fading-memory-optimizer",
 "verification": {
  "benchmark": {
   "track": "dynamics",
   "model": "rnn_small",
   "n_seeds": 8,
   "metric_direction": "lower is better",
   "comparison": {
    "baseline_mean": 0.0024354744236916304,
    "idea_mean": 0.008837972913170233,
    "delta_mean": 0.006402498489478603,
    "per_seed_diffs": [
     -0.001938007539138198,
     0.0036693995352834463,
     0.010719047451857477,
     0.022064058401156217,
     0.0006914960686117411,
     0.0033290251158177853,
     0.0007088985294103622,
     0.011976070352829993
    ],
    "idea_wins": 1,
    "n_pairs": 8,
    "p_value": 0.037,
    "mde": 0.006677981109294924,
    "mde_rel_pct": 274.19631445657353,
    "verdict": "idea worse (significant)",
    "system_worked": false
   },
   "baseline": {
    "best_cfg": {
     "lr": 0.006,
     "momentum": 0.9
    },
    "sweep": [
     {
      "cfg": {
       "lr": 0.001,
       "momentum": 0
      },
      "mean": 0.6811865866184235
     },
     {
      "cfg": {
       "lr": 0.001,
       "momentum": 0.9
      },
      "mean": 0.2693689875304699
     },
     {
      "cfg": {
       "lr": 0.003,
       "momentum": 0
      },
      "mean": 0.5496408864855766
     },
     {
      "cfg": {
       "lr": 0.003,
       "momentum": 0.9
      },
      "mean": 0.005768774193711579
     },
     {
      "cfg": {
       "lr": 0.006,
       "momentum": 0
      },
      "mean": 0.3778964877128601
     },
     {
      "cfg": {
       "lr": 0.006,
       "momentum": 0.9
      },
      "mean": 0.0021511380036827177
     }
    ],
    "full": {
     "mean": 0.0024354744236916304,
     "std": 0.0014542907868172839,
     "per_seed": [
      0.004145679529756308,
      0.002980618504807353,
      0.0008099595434032381,
      0.000668294436763972,
      0.0030060403514653444,
      0.003159687388688326,
      0.004236925859004259,
      0.00047658977564424276
     ],
     "n": 8
    }
   },
   "idea": {
    "mean": 0.008837972913170233,
    "std": 0.006216926394780772,
    "per_seed": [
     0.0022076719906181097,
     0.006650018040090799,
     0.011529006995260715,
     0.02273235283792019,
     0.0036975364200770855,
     0.006488712504506111,
     0.004945824388414621,
     0.012452660128474236
    ],
    "n": 8
   },
   "mechanism_signature": {
    "confirmed": false,
    "observed_decay_rate_mean": -0.10988588097760517,
    "observed_force_ratio_final_mean": 3.349307292855885,
    "observed_force_ratio_final_per_seed": [
     3.3285459078945556,
     3.357568656130888,
     3.3451523205798117,
     3.365962286818286
    ],
    "predicted_decay_rate": 0.006,
    "prediction": "exponential memory state decay rate beta*lr in discrete small-step response"
   },
   "custom_track": null
  },
  "mechanism_confirmed": false,
  "peer_reviewed": false,
  "practical_verdict": "harms",
  "stage1_mechanism_check": {
   "worked": true,
   "confidence": 9,
   "verdict": "Built a self-contained exponential fading-memory optimizer and quadratic verification in run_experiment.py. The continuous characteristic roots satisfy the claimed polynomial to approximately 1e-14 residual, and the predicted and empirical discrete stability limits agree exactly on the tested grid (eta_max=0.63). Fading memory produced zero stiff-mode sign changes versus 249 for SGD and 52 for momentum, demonstrating oscillation suppression, but it converged more slowly at equal steps, so no speedup was observed.",
   "metrics": {
    "baseline": "SGD at eta=0.08: final loss 3.92e-19, 249 stiff-mode sign changes; momentum at eta=0.02: final loss 8.36e-13, 52 sign changes.",
    "idea": "Fading memory with kappa=2, beta=5, eta=0.05: final loss 7.26e-09, 0 stiff-mode sign changes; predicted and empirical stability boundary both 0.63 for lambda=3, kappa=2, beta=5."
   },
   "how_to_run": "python3 run_experiment.py",
   "files": [
    "run_experiment.py",
    "README.md",
    "results.json"
   ],
   "limitations": "Only diagonal quadratic dynamics were tested; no MNIST, Transformer, AdamW integration, wall-clock benchmark, multidimensional non-diagonal Hessian, or broad hyperparameter search was performed. The stability boundary is resolved on a finite eta grid."
  },
  "status": "failed_benchmark",
  "status_label": "Failed on benchmark",
  "updated_at": "2026-09-03T13:13:19",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": false,
    "tested": true
   },
   "practical_benchmark": {
    "beats_baseline": false,
    "tested": true,
    "verdict": "harms"
   },
   "toy_mechanism_gate": {
    "confirmed": true,
    "tested": true
   }
  }
 }
}