# Odd-Drift, Symmetric-Noise Optimizer

- ID: 1463
- Canonical URL: https://synthcore.org/idea/1463/odd-drift-symmetric-noise-optimizer
- API JSON: https://synthcore.org/api/idea/1463.json
- API Markdown: https://synthcore.org/api/idea/1463.md
- Verification status: unverified
- Source: [arXiv:2607.26824](https://arxiv.org/abs/2607.26824)
- Category: dynamics
- Solves: stability, speedup
- ML areas: optimizer, training-dynamics, loss
- Math tags: stochastic-processes, probability, control-theory, dynamical-systems, linear-algebra
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Construct a nonreversible optimizer whose parameter drift contains an antisymmetric mobility component, while its stochastic diffusion and preconditioner remain symmetric positive semidefinite. The paper predicts that adding or removing an antisymmetric diffusion representation cannot change any finite-time joint statistic of scalar state-dependent observables, whereas antisymmetric mobility can change relaxation and response because it enters the drift.

## Mathematical statement

Let \(\theta\in\mathbb{R}^d\) be network parameters, \(L(\theta)\) the loss, \(S=S^{\mathsf T}\succeq0\) a symmetric mobility, and \(A=-A^{\mathsf T}\) an antisymmetric mobility. Use the nonreversible drift \(b(\theta)=-(S+A)\nabla L(\theta)\). The Fokker-Planck operator for a formal diffusion tensor \(D=D_s+D_a\), with \(D_s=D_s^{\mathsf T}\succeq0\) and \(D_a=-D_a^{\mathsf T}\), is \(\partial_t p=-\nabla^{\mathsf T}(bp)+\nabla^{\mathsf T}D\nabla p\). For every smooth scalar function \(g\), \(D_a:\nabla\nabla g=\sum_{ij}(D_a)_{ij}\partial_i\partial_jg=0\) because \(D_a\) is antisymmetric while the Hessian is symmetric. Therefore the generator acting on scalar observables, and consequently arbitrary multipoint state-observable statistics, is unchanged by \(D_a\). The useful trainable mechanism is instead \(A\nabla L\), which changes the first-order drift without injecting an observationally relevant antisymmetric second-order term.

## Key formulas

- $$\partial_t p(\theta,t)=-\nabla_\theta^{\mathsf T}\!\left[b(\theta)p\right]+\nabla_\theta^{\mathsf T}D\nabla_\theta p,\qquad b(\theta)=-(S+A)\nabla_\theta L(\theta),$$
- $$D=D_s+D_a,\quad D_s=D_s^{\mathsf T}\succeq0,\quad D_a=-D_a^{\mathsf T},\qquad D_a:\nabla_\theta\nabla_\theta g=0,$$
- $$\mathrm{d}\theta_t=-(S+A)\nabla L(\theta_t)\,\mathrm{d}t+\sqrt{2D_s}\,\mathrm{d}W_t,$$
- $$\Delta L\approx-\eta\,\nabla L^{\mathsf T}S\nabla L+\frac{\eta^2}{2}\,\nabla L^{\mathsf T}(S+A)^{\mathsf T}H_L(S+A)\nabla L,$$

## Implementation notes

Integrate this mechanism at the optimizer step for an MLP or small Transformer. Maintain a symmetric preconditioner \(S_t\), such as diagonal Adam or RMSProp scaling, and add a cheap antisymmetric block operator \(A_t\). Partition parameters into pairs \((i,j)\) and set \(A_{ij}=a\), \(A_{ji}=-a\) within each pair; use \(a=\alpha/(\sqrt{v_i}+\sqrt{v_j}+\varepsilon)\), where \(v_i\) are Adam second-moment estimates. For each minibatch, compute \(g=\nabla L\), update \(v_t=\beta v_{t-1}+(1-\beta)g^2\), form diagonal \(S_t=(\sqrt{v_t}+\varepsilon)^{-1}\), compute \(A_tg\) using two operations per parameter pair, and update \(\theta\leftarrow\theta-\eta(S_tg+A_tg)+\sqrt{2\eta T S_t}\,\xi\), with \(\xi\sim\mathcal N(0,I)\) and optional temperature \(T\). The paper supplies the exact cancellation of \(D_a\); the preconditioner, temperature, pairing, and \(\alpha\) are engineering choices. First test CIFAR-10 using a two-layer MLP, comparing SGD, Adam, and this optimizer at matched gradient evaluations and wall-clock cost. Record loss, validation accuracy, gradient norms, escape time from a quadratic basin, and lagged joint statistics of loss and gradient norm. The falsifiable prediction is that changing any formal antisymmetric diffusion term leaves scalar-observable statistics unchanged apart from discretization error, whereas changing \(\alpha\) changes relaxation. On a local quadratic \(L(\theta)=\frac12\theta^{\mathsf T}H\theta\), the deterministic update is \(\theta_{k+1}=[I-\eta(S+A)H]\theta_k\); divergence should begin when \(\rho(I-\eta(S+A)H)\ge1\), and the measured threshold should agree within 20 percent.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.