{
 "artifacts": null,
 "category": "dynamics",
 "description": "Represent a small ensemble of parameter vectors or a low-dimensional projection of parameters as a periodically driven probability distribution, and optimize the learning-rate/control waveform so that the distribution translates through successive optimization phases at approximately constant speed without unnecessary reshaping. The method adds a dissipation penalty based on probability-current mismatch and uses automatic differentiation through the particle or density dynamics to learn a periodic optimizer schedule.",
 "formulas_latex": [
  "$$\\dot{x}=\\beta D\\left[f-\\frac{\\partial U(x,\\lambda(t))}{\\partial x}\\right]+\\sqrt{2D}\\,\\eta(t),\\qquad \\partial_t p(x,t)=-\\partial_xJ(x,t),$$",
  "$$J(x,t)=\\beta D\\left[f-\\partial_xU(x,\\lambda(t))\\right]p(x,t)-D\\partial_xp(x,t),\\qquad \\int_0^{2\\pi}p(x,t)\\,dx=1,$$",
  "$$\\mathcal{L}_{\\mathrm{rigid}}=\\frac{1}{T}\\int_0^T\\int\\frac{\\left[J(z,t)-v q(z,t)\\right]^2}{q(z,t)+\\epsilon}\\,dz\\,dt,\\qquad v=\\frac{s(T)-s(0)}{T},$$",
  "$$\\mathcal{J}=\\mathbb{E}[\\ell(\\theta_T)]+\\alpha\\mathcal{L}_{\\mathrm{rigid}}+\\gamma\\frac{1}{T}\\int_0^T\\int\\left(\\partial_tq\\right)^2dz\\,dt,$$"
 ],
 "id": 3135,
 "implementation": "1. Integration point: begin with a cheap two-dimensional experiment. Maintain $N=128$ parameter particles $\\theta_i$ around a central neural-network parameter vector, but project each particle to a scalar coordinate $z_i$ using either loss along a fixed random direction or the first principal component of the particle cloud. Evolve particles for one periodic optimizer cycle with a controllable learning-rate waveform $\\eta(t)$ and optional common drift $b(z,\\lambda)$. Estimate the density $q$ on a periodic grid with a differentiable soft histogram or kernel density estimator. 2. Pseudocode: initialize particles and protocol parameters; for each time step, compute minibatch losses and particle gradients; update $z_i$ using the controlled drift plus noise; estimate $q$ and its spatial derivative; estimate current $J$ by finite differences from the continuity equation or from the particle flux; compute $\\mathcal{J}=\\text{mean loss}+\\alpha\\sum(J-vq)^2/(q+\\epsilon)+\\gamma\\sum(\\partial_tq)^2$; backpropagate through the unrolled cycle and update the protocol parameters. Use Adam for the outer protocol optimization, while comparing against cosine decay, triangular cyclic learning rates, and ordinary SGD/Adam. 3. Computed versus estimated: the Fokker–Planck current formula and rigid-transport penalty are analytic; $q$, $D$, relaxation time, and current are estimated from particles. Do not initially optimize a full neural-network potential; optimize only a Fourier learning-rate waveform $\\eta(t)=\\eta_0+\\sum_{k=1}^K[a_k\\sin(2\\pi kt/T)+b_k\\cos(2\\pi kt/T)]$. 4. First cheap experiment: train a two-layer MLP on MNIST or Fashion-MNIST with 128 particle replicas for 5–20 cycles, using identical minibatch seeds across schedules. Measure endpoint loss, particle-cloud covariance, current mismatch, and cycle-to-cycle loss. The quantitative prediction is a sharp degradation boundary when commanded phase speed $v$ becomes comparable to the intrinsic relaxation speed $L/\\tau_{\\mathrm{relax}}$, with $\\tau_{\\mathrm{relax}}\\approx L^2/D$ measured from an uncontrolled perturbation experiment. Below this boundary, $\\mathcal{L}_{\\mathrm{rigid}}$ and covariance deformation should remain approximately flat; above it they should increase rapidly, approximately as a power law in $v\\tau_{\\mathrm{relax}}/L$. A successful implementation should shift the boundary upward by at least 20% relative to an unregularized cyclic schedule while reaching equal loss with lower gradient variance.",
 "math_summary": "The paper models an overdamped periodic coordinate $x\\in[0,2\\pi)$ by $\\dot{x}=\\beta D[f-\\partial_x U(x,\\lambda(t))]+\\sqrt{2D}\\,\\eta(t)$, where $\\beta=(k_{\\mathrm B}T)^{-1}$, $D$ is the diffusion coefficient, $f$ is a constant nonequilibrium force, $U$ is a controlled potential, $\\lambda(t)$ is the protocol, and $\\eta$ is unit white noise. The corresponding density $p(x,t)$ obeys $\\partial_t p=-\\partial_xJ$ with probability current $J=\\beta D(f-\\partial_xU)p-D\\partial_xp$. For an ML ensemble, let $z$ be a one-dimensional phase or loss-coordinate projection of parameter particles, let $q(z,t)$ be their density, let $V(z,\\lambda(t))$ be a trainable control potential, and let $b$ be a drift induced by the optimizer. Use $\\partial_tq=-\\partial_zJ$ and $J=[b-\\mu\\partial_zV]q-D\\partial_zq$, where $\\mu$ is mobility. The constant-speed target is rigid transport $q(z,t)\\approx q_0(z-s(t))$ with $\\dot{s}(t)=v$ constant; its ideal current is $J_{\\mathrm{rigid}}=vq$. Penalizing $\\int(J-vq)^2/(q+\\epsilon)\\,dz\\,dt$ suppresses density deformation and excess stochastic transport. The predicted stability signature is that deformation and gradient variance grow sharply when the cycle duration is too short relative to the relaxation time $\\tau_{\\mathrm{relax}}\\sim L^2/D$ or when the commanded speed exceeds the density's transport capacity.",
 "math_tags": [
  "stochastic-processes",
  "control-theory",
  "dynamical-systems",
  "optimization",
  "pde"
 ],
 "ml_areas": [
  "optimizer",
  "training-dynamics",
  "scheduler"
 ],
 "paper": {
  "arxiv_id": "2609.03217",
  "arxiv_url": "https://arxiv.org/abs/2609.03217",
  "summary_what_math_gives_to_ml": "The paper provides a constructive mechanism for optimizing periodically driven nonequilibrium systems: automatic differentiation through a Fokker–Planck solver is used to optimize a control protocol, with low mechanical dissipation achieved by rotating the entire probability distribution at nearly constant angular speed while preserving its shape. This suggests treating a population of neural-network parameters, particles, or latent states as a periodically driven probability density and optimizing the learning-rate/control waveform against a dissipation-like objective rather than only endpoint loss. The most promising transfer is a distributional optimizer whose periodic cycles maintain a nearly rigidly translating parameter distribution, with deviations from constant-speed transport serving as a measurable instability and schedule-adjustment signal.",
  "title": "Optimal Control of Periodic Nonequilibrium Mechanochemical Systems via Automatic Differentiation",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 8,
  "usefulness": 6
 },
 "solves": [
  "stability",
  "speedup",
  "generalization"
 ],
 "title": "Constant-Speed Distributional Optimizer",
 "url": "https://synthcore.org/idea/3135/constant-speed-distributional-optimizer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}