Unverified 2026

Dynamic-scaling cyclic optimizer

Implementation & benchmark of arXiv:2608.05936 — Dynamic scaling behavior in the presence of a periodic magnetic driving across Ising continuous transitions

Usefulness5/10

Difficulty4/10

Novelty8/10

Training Stability Speedup Optimizer Scheduler Training dynamics Statistical mechanics Dynamical systems Renormalization

Source paper: Dynamic scaling behavior in the presence of a periodic magnetic driving across Ising continuous transitions arXiv:2608.05936 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Drive the optimizer periodically around a baseline learning rate, but scale the modulation amplitude and period through a single dimensionless control variable rather than tuning them independently. The neural analogue predicts that normalized loss, gradient norm, and parameter-displacement trajectories should approximately collapse across schedules with equal $aP^{\kappa}$, while sufficiently large values should reveal a measurable transition from weak tracking to strongly oscillatory or unstable training.

Formulas

$$h(t)=-A\cos\!\left(\frac{2\pi t}{P}\right),\qquad \tau=\frac{t}{P},\qquad \sigma=AP^{\kappa},\qquad \kappa=\frac{y_h}{z},\qquad y_h=\frac{d+2-\eta}{2}.$$

$$X(t;A,P)\simeq \mathcal{F}\!\left(\tau,\sigma\right)=\mathcal{F}\!\left(\frac{t}{P},AP^{\kappa}\right),$$

$$\eta_n=\eta_0\left[1+a\cos\!\left(\frac{2\pi n}{P}\right)\right],\qquad \sigma_{\mathrm{NN}}=aP^{\kappa_{\mathrm{NN}}},$$

$$\mathcal{E}(\sigma)=\operatorname{Var}_{\text{seeds},\text{phase}}\!\left[\widetilde X(n;a,P)-\widetilde X\!\left(n;a',P'\right)\right],\quad aP^{\kappa_{\mathrm{NN}}}=a'P'^{\kappa_{\mathrm{NN}}}.$$

Mathematical statement

For the periodically driven Ising system, the forcing is $h(t)=-A\cos(2\pi t/P)$, where $A$ is amplitude, $P$ is period, and $t$ is time. At criticality the independent scaling variables are $\tau=t/P$ and $\sigma=AP^{\kappa}$, with $\kappa=y_h/z$, $y_h=(d+2-\eta)/2$, spatial dimension $d$, anomalous exponent $\eta$, and dynamic exponent $z$. Transfer this structure to optimizer modulation $\eta_n=\eta_0[1+a\cos(2\pi n/P)]$, where $n$ is the update index, $\eta_0$ is the baseline learning rate, and $a$ is dimensionless modulation amplitude. Define $\sigma_{\mathrm{NN}}=aP^{\kappa_{\mathrm{NN}}}$. The exponent $\kappa_{\mathrm{NN}}$ is estimated from the unforced optimizer's relaxation law, or swept empirically. The testable prediction is that an observable trajectory $X(n;a,P)$, after normalization, is approximately $F(n/P,\sigma_{\mathrm{NN}})$, so equal $\sigma_{\mathrm{NN}}$ schedules have similar phase-resolved dynamics.

Implementation notes

1. Integration point: replace a constant learning rate in SGD or Adam with a multiplicatively modulated schedule, keeping the optimizer state update unchanged. Use $\eta_n=\eta_0[1+a\cos(2\pi n/P)]$, clip $\eta_n$ to a positive interval, and initially hold momentum, weight decay, batch size, and data order fixed. Monitor loss, gradient norm, parameter displacement, update-to-gradient ratio, and validation accuracy as phase-resolved observables. 2. Pseudocode: choose $\eta_0,a,P$; at update $n$, compute $\eta_n$; compute minibatch gradient $g_n$; perform the baseline Adam or SGD update using $\eta_n$; record $X_n$. Repeat for pairs $(a,P)$ satisfying $aP^{\kappa_{NN}}=\sigma$. Estimate $\kappa_{NN}$ by fitting the decay time of an unforced perturbation response, or sweep $\kappa_{NN}\in[0,2]$. 3. Computed from the paper: the dimensionless combination and phase variable. Estimated empirically: the neural exponent, response-collapse error, and instability threshold. 4. First cheap experiment: train a 2-layer MLP on MNIST with SGD and compare $P\in\{20,40,80,160\}$, choosing $a=\sigma P^{-\kappa_{NN}}$ at fixed $\sigma$, against constant learning rate and cosine annealing. Plot phase-aligned normalized trajectories. The quantitative prediction is that equal-$\sigma$ curves have lower collapse error than equal-amplitude curves; require at least a 30% reduction in collapse error. Increase $\sigma$ until update norms grow persistently; the oscillatory or divergence boundary should be approximately a single threshold in $\sigma$, with less than 20% variation across periods.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.