Response-Calibrated Langevin Optimizer

1. Integration point: wrap stochastic SGD after gradient computation and preconditioning but before the parameter update. Choose $A$ as the average training loss over a window of $m$ steps, or as $A=\sum_{j=0}^{m-1}u^\top g_{k+j}$ for a fixed random unit vector $u$. Every $K$ steps, launch two synchronized short trajectories from the same checkpoint and feed them identical minibatches and Gaussian random numbers. The perturbed trajectory receives a drift perturbation $hq_k$, where $q_k$ is the current gradient or a low-rank random direction. A separate test can perturb the diffusion scale as $D_k\mapsto D_k(1+h)$, which is important because the paper treats perturbations of both drift and noise. 2. Pseudocode: run baseline and perturbed replicas; estimate $r=(\bar A_h-\bar A_0)/h$; compute the Gaussian transition-density path score $S_h$ analytically; estimate $c=\operatorname{Cov}(A,S_h)$ and $v=\operatorname{Var}(S_h)$; form $R=r-c$; update $\log D\leftarrow\log D-\alpha R/(\sqrt v+\epsilon)$ with clipping to $D\in[D_{\min},D_{\max}]$. 3. Compute the score from the implemented transition kernel, while estimating responses, covariances, and variances with exponential moving averages. Use only scalar score projections, not the full parameter-dimensional path score. 4. First experiment: train a small MLP on MNIST with isotropic Langevin SGD, comparing fixed noise, cosine-decayed noise, and response-calibrated noise. Use $m=20$, $K=100$, and $h=10^{-3}$ relative to update magnitude. Measure accuracy, loss variance, and $|R|/(|c|+\epsilon)$. The quantitative prediction is that calibration reduces this residual and that $r^2\leq\operatorname{Var}(A)v$ holds within sampling error; increasing $D$ past the near-saturation point should increase loss variance and eventually cause an observable instability transition.