{
 "artifacts": null,
 "category": "dynamics",
 "description": "Treat a slowly varying block of neural-network parameters as a coarse-grained stochastic process and continuously estimate both its covariance spectrum and its linear response to small artificial perturbations. Use the fluctuation–response mismatch as a feedback signal to tune injected parameter noise or minibatch size; the thermal Einstein relation is imposed only when a calibrated equilibrium-like regime is desired, while antisymmetric response components are retained as admissible nonequilibrium probability currents.",
 "formulas_latex": [
  "$$d\\theta_t=b(\\theta_t)\\,dt+B\\,dW_t,\\qquad D=\\frac{1}{2}BB^{\\mathsf T}.$$",
  "$$R_{ij}(t)=\\left.\\frac{\\delta\\,\\mathbb{E}[\\theta_i(t)]}{\\delta h_j(0)}\\right|_{h=0},\\qquad C_{ij}(\\omega)=\\left\\langle\\delta\\theta_i(\\omega)\\delta\\theta_j(-\\omega)\\right\\rangle.$$",
  "$$C(\\omega)=\\frac{2T}{\\omega}\\operatorname{Im}R(\\omega),\\qquad E(\\omega)=C(\\omega)-\\frac{2T}{\\omega}\\operatorname{Im}R(\\omega).$$",
  "$$d\\theta=-A\\theta\\,dt+B\\,dW_t\\quad\\Longrightarrow\\quad A\\Sigma+\\Sigma A^{\\mathsf T}=BB^{\\mathsf T},\\qquad \\operatorname{Re}\\lambda_k(A)\u003e0.$$"
 ],
 "id": 2745,
 "implementation": "Integrate the mechanism at the optimizer level, initially on a small MLP or one transformer block rather than the full parameter vector. Maintain an orthonormal random projection matrix \\(U\\in\\mathbb{R}^{p\\times r}\\), with \\(r=8\\) to \\(32\\), and record projected parameter coordinates \\(z_t=U^{\\mathsf T}(\\theta_t-\\bar\\theta)\\) every optimizer step. Every \\(K=100\\) to \\(500\\) steps, create a shadow copy of the optimizer state and apply a one-step impulse \\(h u\\) to the projected update, using a fresh random direction \\(u\\). Run perturbed and control trajectories for a short horizon and estimate \\(\\widehat R(\\tau)=[z^{\\mathrm{pert}}_{t+\\tau}-z^{\\mathrm{control}}_{t+\\tau}]/h\\). Estimate \\(\\widehat C(\\omega)\\) from a windowed FFT of the recorded \\(z_t\\). Fit \\(\\widehat T\\) by least squares over low frequencies, then compute the scalar mismatch \\(m=\\sum_\\omega\\|\\widehat C(\\omega)-2\\widehat T\\operatorname{Im}\\widehat R(\\omega)/\\omega\\|_F^2\\). Adapt the injected noise scale \\(q\\) using \\(\\log q\\leftarrow\\log q-\\alpha(m-m_0)\\), clipped to preset bounds; alternatively adapt minibatch size to control gradient-noise variance. The paper supplies the Ward/FDT structure, but response, covariance, stationarity, and temperature are estimated empirically. First experiment: train an MLP on MNIST with SGD, SGD plus fixed Langevin noise, and the adaptive controller. Use \\(r=16\\), \\(K=200\\), and several impulse sizes. The predicted signature is a measurable reduction of low-frequency mismatch \\(E(\\omega)\\) by at least 30% in the calibrated regime. In a linear quadratic toy model, the stability boundary must remain \\(\\operatorname{Re}\\lambda_k(A)=0\\): noise adaptation can change covariance but cannot stabilize an unstable drift.",
 "math_summary": "Let \\(\\theta_t\\in\\mathbb{R}^d\\) denote parameters or a low-dimensional projection of parameters, modeled by \\(d\\theta=b(\\theta)dt+B\\,dW_t\\), where \\(b\\) is drift, \\(W_t\\) is Brownian motion, and \\(D=BB^{\\mathsf T}/2\\) is the diffusion tensor. Apply a small impulse \\(h(t)u\\) to the update and estimate the retarded response \\(R(t)=\\delta\\langle\\theta(t)\\rangle/\\delta h(0)\\), with Fourier transform \\(R(\\omega)\\). Let \\(C(\\omega)=\\langle\\delta\\theta(\\omega)\\delta\\theta(-\\omega)^{\\mathsf T}\\rangle\\) be the symmetric fluctuation spectrum. In the high-temperature Einstein/FDT limit, the Ward identity becomes \\(C(\\omega)=2T\\operatorname{Im}R(\\omega)/\\omega\\), where \\(T\\) is the effective noise temperature. Away from this limit, the mismatch \\(E(\\omega)=C(\\omega)-2T\\operatorname{Im}R(\\omega)/\\omega\\) measures nonequilibrium behavior and should not automatically be penalized. For a linearized optimizer \\(d\\theta=-A\\theta dt+B dW\\), stationarity requires \\(\\operatorname{Re}\\lambda_k(A)\u003e0\\), and the covariance satisfies the Lyapunov equation \\(A\\Sigma+\\Sigma A^{\\mathsf T}=BB^{\\mathsf T}\\).",
 "math_tags": [
  "stochastic-processes",
  "statistical-mechanics",
  "dynamical-systems",
  "control-theory",
  "probability"
 ],
 "ml_areas": [
  "optimizer",
  "training-dynamics",
  "scheduler",
  "regularization"
 ],
 "paper": {
  "arxiv_id": "2608.26468",
  "arxiv_url": "https://arxiv.org/abs/2608.26468",
  "summary_what_math_gives_to_ml": "The paper derives fluctuation–response Ward identities from an emergent \\(\\mathbb{Z}_2\\) symmetry of a coarse-grained stochastic action, without assuming detailed balance or thermal equilibrium. Its transferable asset is a practical distinction between response relations fixed by stochastic dynamics and the additional Einstein relation needed to determine the absolute noise scale. In neural-network training, this suggests an online noise-calibration controller that measures how parameter statistics respond to small perturbations and adjusts minibatch noise or injected Langevin noise toward a target fluctuation–response relation, while allowing nonequilibrium circulating dynamics rather than forcing reversible optimization.",
  "title": "Fluctuation--response relations from an emergent $\\mathbb{Z}_2$ symmetry in the rotating stochastic Landau model",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 7,
  "usefulness": 6
 },
 "solves": [
  "stability",
  "generalization",
  "accuracy"
 ],
 "title": "Ward-Calibrated Training Noise",
 "url": "https://synthcore.org/idea/2745/ward-calibrated-training-noise",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}