{
 "artifacts": [
  {
   "name": "experiment.py",
   "url": "https://synthcore.org/code/1214/experiment.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1214/report.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/1214/results.json"
  }
 ],
 "category": "dynamics",
 "description": "Treat local neural-network training as a driven linear system and periodically modulate the learning rate by a small sinusoid. Estimate the transfer function from this modulation to loss or gradient observables, fit its relaxation poles, and set the learning rate below the measured instability boundary.",
 "download_zip": "https://synthcore.org/download/1214",
 "formulas_latex": [
  "$$\\dot{x}=Ax+Bu,\\qquad y=Cx$$",
  "$$H(i\\omega)=\\frac{\\widehat{y}(\\omega)}{\\widehat{u}(\\omega)}=C(i\\omega I-A)^{-1}B=\\sum_{k=1}^{N}\\frac{a_k}{1+i\\omega\\tau_k},\\qquad \\tau_k=-\\lambda_k^{-1}\u003e0$$",
  "$$z_{k,t+1}=(1-\\eta\\lambda_k)z_{k,t},\\qquad |1-\\eta\\lambda_k|\u003c1\\Longleftrightarrow 0\u003c\\eta\u003c\\frac{2}{\\lambda_k}$$",
  "$$\\eta_{t+1}=\\operatorname{clip}\\left(\\eta_t\\min\\left(1.1,\\frac{2\\gamma}{\\eta_t\\widehat{\\lambda}_{\\max}}\\right),\\eta_{\\min},\\eta_{\\max}\\right),\\qquad 0\u003c\\gamma\u003c1$$"
 ],
 "id": 3060,
 "implementation": "Integrate this as a wrapper around SGD, initially avoiding Adam so the identified dynamics remain interpretable. Every K optimizer steps, apply a small learning-rate probe \\(u_t=\\epsilon\\sin(\\omega t)\\) for several frequencies spanning approximately one decade below and above the estimated inverse relaxation time. Record minibatch loss, gradient norm, and optionally a random projection of parameter displacement. For each frequency, estimate the complex response with lock-in correlations, \\(\\widehat H(\\omega)=2\\langle y_t e^{-i\\omega t}\\rangle/(\\epsilon T)\\), over an integer number of probe periods. Fit one to four positive time constants and residues by nonlinear least squares. The paper-derived quantities are the response poles and RC-style decomposition; the Jacobian, residues, and effective curvatures are estimated empirically. Infer \\(\\widehat{\\lambda}_{\\max}=2/\\widehat{\\eta}_{\\mathrm{crit}}\\) from the largest learning rate whose oscillations still decay, then operate at \\(\\gamma\\widehat{\\eta}_{\\mathrm{crit}}\\), with \\(\\gamma=0.7\\). First experiment: train a three-layer MLP on MNIST and compare fixed SGD, cosine decay, and impedance control at equal update count. Sweep fixed learning rates around the predicted boundary. The falsifiable signature is a sharp transition from decaying to growing or alternating loss oscillations near \\(\\eta_c\\); the impedance estimate should predict \\(\\eta_c\\) within 20 percent, while controlled training should remain below the boundary and reduce settling time.",
 "math_summary": "Linearize training dynamics as \\(\\dot{x}=Ax+Bu\\), where \\(x\\) is parameter displacement, \\(A\\) is the local training Jacobian, \\(u\\) is a scalar learning-rate modulation, and \\(B\\) is its input direction. For observable \\(y=Cx\\), with observation row vector \\(C\\), the frequency response is \\(H(i\\omega)=C(i\\omega I-A)^{-1}B\\). For real stable modes, \\(H(i\\omega)=\\sum_k a_k/(1+i\\omega\\tau_k)\\), where \\(\\tau_k=-1/\\lambda_k\u003e0\\) are relaxation times, \\(\\lambda_k\\) are stable eigenvalues, and \\(a_k\\) are residues. For a quadratic local loss, discrete gradient descent has mode update \\(z_{k,t+1}=(1-\\eta\\lambda_k)z_{k,t}\\), stable when \\(0\u003c\\eta\u003c2/\\lambda_k\\).",
 "math_tags": [
  "dynamical-systems",
  "control-theory",
  "spectral-theory",
  "linear-algebra"
 ],
 "ml_areas": [
  "optimizer",
  "scheduler",
  "training-dynamics"
 ],
 "paper": {
  "arxiv_id": "2609.02458",
  "arxiv_url": "https://arxiv.org/abs/2609.02458",
  "summary_what_math_gives_to_ml": "The paper gives a constructive linear-response description of periodically driven finite-state stochastic dynamics: each current response can be represented by an equivalent circuit with N resistor-capacitor series branches. The transferable mechanism is a frequency-dependent susceptibility whose poles encode relaxation times, with low-frequency quasi-static behavior and high-frequency attenuation. In neural-network training, this can become online system identification: periodically perturb the learning rate or update magnitude, estimate the optimizer-to-loss transfer function, and use its poles to select damping and step sizes before oscillatory or unstable modes dominate.",
  "title": "Impedance in Periodically Driven Stochastic Systems",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 7,
  "usefulness": 8
 },
 "solves": [
  "stability",
  "speedup"
 ],
 "title": "Impedance-Calibrated Learning-Rate Control",
 "url": "https://synthcore.org/idea/3060/impedance-calibrated-learning-rate-control",
 "verification": {
  "peer_reviewed": false,
  "stage1_mechanism_check": {
   "worked": false,
   "confidence": 8,
   "verdict": "Built a reproducible lock-in transfer-function sanity check, quadratic stability test, and small CUDA digits-MLP comparison in experiment.py. The core math was confirmed: corrected lock-in recovery had 0.82% relative error, and the quadratic transition occurred at the predicted eta=2/lambda_max=0.4. The training controller showed a promising result versus the selected fixed and cosine baselines, with test loss 0.0755 versus 0.1098 and 0.1616, but this is not a demonstrated impedance-control win because the implementation used Hessian power estimates rather than fitted sinusoidal poles/residues, and fixed-rate eta=0.6 performed better.",
   "metrics": {
    "baseline": "Fixed SGD eta=0.08: test loss 0.1098, accuracy 97.78%; cosine SGD: test loss 0.1616, accuracy 96.67%; fixed-rate eta=0.6 reached sampled final loss 0.0265.",
    "idea": "Curvature-controlled SGD: test loss 0.0755, accuracy 98.00%, final learning rate 0.35; lock-in transfer relative error 0.0082; predicted critical eta 0.400, grid estimate 0.395; quadratic eta=0.39 decayed while eta=0.41 and 0.50 grew."
   },
   "how_to_run": "python3 experiment.py",
   "files": [
    "experiment.py",
    "results.json",
    "run_output.txt"
   ],
   "limitations": "Only one random seed and one small sklearn digits dataset were tested. The real-training prototype does not implement the proposed multi-frequency probing, pole/residue nonlinear fit, or loss/gradient lock-in estimator; it estimates curvature directly with a short Hessian-vector power iteration. No FLOP-normalized or repeated-seed comparison, larger model, MNIST, or robust instability-boundary sweep was performed."
  },
  "status": "mechanism_failed",
  "status_label": "Mechanism failed",
  "updated_at": "2026-09-03T12:03:14",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": false,
    "tested": true
   }
  }
 }
}