# Convolution-Calibrated Persistent-Noise Optimizer

- ID: 2679
- Canonical URL: https://synthcore.org/idea/2679/convolution-calibrated-persistent-noise-optimizer
- API JSON: https://synthcore.org/api/idea/2679.json
- API Markdown: https://synthcore.org/api/idea/2679.md
- Verification status: unverified
- Source: [arXiv:2608.21781](https://arxiv.org/abs/2608.21781)
- Category: dynamics
- Solves: stability, generalization, sample-efficiency
- ML areas: optimizer, training-dynamics, regularization
- Math tags: statistical-mechanics, stochastic-processes, dynamical-systems, probability, optimization
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Add a persistent two-state force to a locally stable optimizer while retaining Gaussian minibatch or Langevin noise. In a locally quadratic basin, the parameter-error distribution should be the convolution of a compact-support run-and-tumble stationary law and an Ornstein-Uhlenbeck Gaussian. This supplies an explicit persistence and noise calibration rule instead of treating all optimizer noise as white and Gaussian.

## Mathematical statement

For one parameter direction with local curvature lambda greater than zero, use dtheta/dt = -mu lambda theta + v sigma(t) + xi(t), where theta is parameter error, mu is positive mobility, v is persistent-force magnitude, sigma belongs to {-1,+1} and flips at rate alpha/2, and xi is Gaussian white noise with covariance 2 D_T delta(t-t'). The athermal stationary law has support |theta| less than a, with a = v/(mu lambda), and density proportional to [1-(theta/a)^2]^(r-1), where r = alpha/(2 mu lambda). Thermal noise contributes the Ornstein-Uhlenbeck kernel with variance s_T squared = D_T/(mu lambda). The complete stationary law is their convolution. The dimensionless ratios r and chi = D_T/(v^2/alpha) predict the shape crossover: small r and chi produce boundary-peaked non-Gaussian fluctuations, whereas large r or chi produces a Gaussian-like distribution.

## Key formulas

- $$\frac{d\theta}{dt}=-\mu\lambda\theta+v\sigma(t)+\xi(t),\qquad \sigma\in\{-1,+1\},\qquad \Pr[\text{flip in }dt]=\frac{\alpha}{2}dt$$
- $$p_{\mathrm{RT}}(\theta)=\frac{[1-(\theta/a)^2]^{r-1}}{aB(1/2,r)}\mathbf{1}_{|\theta|<a},\qquad a=\frac{v}{\mu\lambda},\qquad r=\frac{\alpha}{2\mu\lambda}$$
- $$p(\theta)=\int_{-a}^{a}p_{\mathrm{RT}}(y)G_T(\theta-y)\,dy,\qquad G_T(z)=\frac{1}{\sqrt{2\pi s_T^2}}\exp\left(-\frac{z^2}{2s_T^2}\right),\qquad s_T^2=\frac{D_T}{\mu\lambda}$$
- $$\operatorname{Var}_{\mathrm{RT}}(\theta)=\frac{a^2}{2r+1},\qquad \chi=\frac{D_T}{v^2/\alpha}$$

## Implementation notes

Integrate the mechanism into SGD, Adam, or Langevin SGD inside a trust region, initially on one parameter block or a low-dimensional projection. Estimate a local curvature lambda using a Hessian-vector product, diagonal Gauss-Newton estimate, or finite-difference curvature. Maintain one persistent sign sigma per block. At step k with duration h, flip sigma with probability 1 - exp(-alpha h/2), compute the minibatch gradient g, and update theta by theta <- theta + h(-mu g + v sigma) + sqrt(2 D_T h) epsilon, where epsilon is standard normal noise. Use gradient clipping or a trust-region projection to prevent the persistent force from leaving the local basin. The paper provides the stationary run-and-tumble density and its Ornstein-Uhlenbeck convolution; curvature, effective thermal noise D_T, and the validity of the quadratic approximation must be estimated from training trajectories. The first cheap experiment should use a one-dimensional and two-dimensional quadratic loss, followed by a small MLP on MNIST. Compare SGD, Gaussian Langevin SGD, and persistent-noise SGD at matched total variance. Record parameter histograms after equilibration and fit both the predicted convolution and a Gaussian. The quantitative prediction is that the convolution fit wins at small r or chi, while the histogram becomes Gaussian-like when r or chi is large. The measured variance should match a^2/(2r+1) + D_T/(mu lambda) within 20 percent. For the discrete deterministic part, instability should begin when |1-h mu lambda| reaches one, with the upper boundary h mu lambda = 2.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.