{
 "artifacts": null,
 "category": "sampling",
 "description": "Train multiple neural-network parameter particles with a deterministic Stein interaction plus Langevin noise instead of using independent SGD or SGLD chains. The Stein term repels nearby particles while moving the ensemble toward high target probability, and the Langevin term supplies diffusion that improves exploration and prevents particle collapse.",
 "formulas_latex": [
  "$$v_{\\rho}(x):=\\int_{\\mathbb{R}^{d}}\\{-k(x,y)\\nabla V(y)+\\nabla_{2}k(x,y)\\}\\,\\rho(dy),\\qquad \\partial_t\\mu_t=-\\nabla\\cdot(\\mu_t v_{\\mu_t})$$",
  "$$\\frac{d}{dt}\\mathrm{KL}(\\mu_t\\|\\pi)=-\\mathrm{KSD}^{2}(\\mu_t\\|\\pi),\\qquad \\mathrm{KSD}(\\rho\\|\\pi)=\\|v_{\\rho}\\|_{\\mathcal H^{d}}$$",
  "$$\\frac{d}{dt}\\mathrm{KL}(\\mu_t\\|\\pi)=-\\mathrm{KSD}^{2}(\\mu_t\\|\\pi)-\\mathcal I(\\mu_t\\|\\pi),\\qquad \\mathcal I(\\mu\\|\\pi)=\\int\\left\\|\\nabla\\log\\frac{d\\mu}{d\\pi}\\right\\|^{2}d\\mu$$",
  "$$\\theta_i^{t+1}=\\theta_i^t+\\eta\\frac{1}{N}\\sum_{j=1}^{N}\\left[-k(\\theta_i^t,\\theta_j^t)\\nabla V(\\theta_j^t)+\\nabla_{\\theta_j}k(\\theta_i^t,\\theta_j^t)\\right]+\\sqrt{2\\eta T}\\,\\xi_i^t,\\qquad \\xi_i^t\\sim\\mathcal N(0,I)$$"
 ],
 "id": 2793,
 "implementation": "Integrate the method at the optimizer or sampler level by maintaining \\(N\\) complete neural-network parameter vectors \\(\\theta_1,\\ldots,\\theta_N\\). Let \\(V(\\theta)\\) be the negative log posterior, or a scaled training loss plus a prior penalty. At every step, use the same minibatch for all particles so the Stein interaction sees comparable stochastic gradients. Compute \\(g_j=\\nabla_\\theta V(\\theta_j)\\), pairwise kernels \\(K_{ij}=k(\\theta_i,\\theta_j)\\), and kernel derivatives. For an RBF kernel, use \\(K_{ij}=\\exp(-\\|\\theta_i-\\theta_j\\|^2/(2h^2))\\) and \\(\\nabla_{\\theta_j}K_{ij}=K_{ij}(\\theta_i-\\theta_j)/h^2\\). Pseudocode: compute all \\(g_j\\); for each \\(i\\), initialize \\(v_i=0\\); for each \\(j\\), add \\((-K_{ij}g_j+\\nabla_{\\theta_j}K_{ij})/N\\); update \\(\\theta_i\\leftarrow\\theta_i+\\eta v_i+\\sqrt{2\\eta T}\\xi_i\\). Here \\(T\\) is the temperature and \\(\\xi_i\\) is standard Gaussian noise. Estimate bandwidth \\(h\\) from the median pairwise distance, optionally using a random parameter projection. Compute empirical KSD from the RKHS velocity norm and track particle diversity; these are diagnostics, while the entropy identity is the mathematical motivation rather than a directly observed quantity. First test on a two-layer MLP on MNIST with \\(N=8\\) or \\(16\\), comparing independent Adam, pure SVGD, SGLD, and the hybrid at equal gradient-evaluation budgets. Measure test NLL, accuracy, calibration error, empirical KSD, pairwise output diversity, and particle-collapse rate. Success means lower test NLL or calibration error at matched compute, stable last-iterate performance, and larger useful diversity without uncontrolled loss variance.",
 "math_summary": "For a target density \\(\\pi(dx)\\propto e^{-V(x)}dx\\), define the mean-field Stein velocity \\(v_{\\rho}(x)=\\int[-k(x,y)\\nabla V(y)+\\nabla_{2}k(x,y)]\\,\\rho(dy)\\), where \\(k\\) is a positive-definite scalar kernel, \\(V\\) is the negative log target density, and \\(\\nabla_{2}k\\) differentiates the second kernel argument. The deterministic continuity equation \\(\\partial_t\\mu_t=-\\nabla\\cdot(\\mu_t v_{\\mu_t})\\) satisfies \\(\\frac{d}{dt}\\mathrm{KL}(\\mu_t\\|\\pi)=-\\mathrm{KSD}^{2}(\\mu_t\\|\\pi)\\), where \\(\\mathrm{KSD}(\\rho\\|\\pi)=\\|v_\\rho\\|_{\\mathcal H^d}\\) and \\(\\mathcal H^d\\) is the vector-valued RKHS generated by \\(k\\). Adding Langevin diffusion contributes relative Fisher-information dissipation, giving the adapted identity \\(\\frac{d}{dt}\\mathrm{KL}(\\mu_t\\|\\pi)=-\\mathrm{KSD}^{2}(\\mu_t\\|\\pi)-\\mathcal I(\\mu_t\\|\\pi)\\), with \\(\\mathcal I(\\mu\\|\\pi)=\\int\\|\\nabla\\log(d\\mu/d\\pi)\\|^2d\\mu\\). Under a log-Sobolev inequality \\(\\mathrm{KL}(\\mu\\|\\pi)\\leq(2\\lambda)^{-1}\\mathcal I(\\mu\\|\\pi)\\), the Langevin component guarantees exponential entropy decay at rate at least \\(2\\lambda\\), while the Stein term adds nonnegative dissipation without requiring Euclidean contractivity.",
 "math_tags": [
  "probability",
  "information-theory",
  "optimal-transport",
  "functional-analysis"
 ],
 "ml_areas": [
  "optimizer",
  "sampling",
  "regularization"
 ],
 "paper": {
  "arxiv_id": "2608.28827",
  "arxiv_url": "https://arxiv.org/abs/2608.28827",
  "summary_what_math_gives_to_ml": "The paper provides an entropy-dissipation framework for particle systems combining deterministic Stein transport with Langevin diffusion. The transferable asset is that both components reduce the same relative entropy through distinct geometries: kernel Stein discrepancy for repulsive transport and relative Fisher information for stochastic diffusion. This suggests a particle-based neural Bayesian or ensemble-training optimizer in which repulsion improves coverage while Langevin noise prevents collapse, with KSD and gradient-noise proxies used to adapt their balance. The long-time analysis also motivates evaluating last-iterate performance rather than relying only on finite-horizon averages.",
  "title": "Quantitative Target Convergence and Uniform-in-Time Propagation of Chaos for Langevin-Regularized SVGD",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 5,
  "usefulness": 6
 },
 "solves": [
  "accuracy",
  "stability",
  "sample-efficiency"
 ],
 "title": "Entropy-dissipating Stein-Langevin particle optimizer",
 "url": "https://synthcore.org/idea/2793/entropy-dissipating-stein-langevin-particle-optimizer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}