# FDT-Calibrated Rotational Optimizer

- ID: 1413
- Canonical URL: https://synthcore.org/idea/1413/fdt-calibrated-rotational-optimizer
- API JSON: https://synthcore.org/api/idea/1413.json
- API Markdown: https://synthcore.org/api/idea/1413.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.25782](https://arxiv.org/abs/2607.25782)
- Category: dynamics
- Solves: stability, speedup, accuracy
- ML areas: optimizer, training-dynamics, scheduler
- Math tags: statistical-mechanics, stochastic-processes, dynamical-systems, control-theory, linear-algebra
- Ratings: usefulness 7/10; difficulty 6/10; novelty 7/10

## Idea description

Add a controlled antisymmetric component to the local parameter update so optimization can circulate around ill-conditioned valleys instead of moving only along gradient directions. The symmetric component supplies dissipation, while the skew component produces the oscillatory non-reciprocal response predicted by the paper. Adapt the skew strength only while the estimated discrete-time dynamics remain stable.

## Mathematical statement

Approximate a parameter block near its current state by the noisy linear dynamics d x/dt = J x + square-root(2D) xi(t), where x is a centered parameter displacement, J is the local Jacobian, D is gradient-noise covariance, and xi is white noise. Decompose J = -S + A, where S = S-transpose is the dissipative symmetric part and A = -A-transpose is the antisymmetric part. Continuous-time stability requires every eigenvalue of J to have negative real part; Euler updates require spectral radius rho(I + eta J) less than one, where eta is the learning rate. The stationary covariance C obeys J C + C J-transpose + 2D = 0. The impulse response is R(t) = exp(J t), while an equilibrium fluctuation-response relation would approximately connect R(t) to minus beta times the time derivative of C(t). Their mismatch measures broken detailed balance. For J = -s I + a K with K antisymmetric, eigenvalues are -s plus or minus i a, giving decay rate s and oscillation frequency absolute value of a.

## Key formulas

- $$\dot{x}=(-S+A)x+\sqrt{2D}\,\xi(t),\qquad S=S^{T}\succeq 0,\quad A=-A^{T}$$
- $$\rho(I+\eta J)<1,\qquad \max_k\operatorname{Re}\lambda_k(J)<0$$
- $$JC+CJ^{T}+2D=0,\qquad R(t)=e^{Jt}$$
- $$J=-sI+aK,\quad K^{T}=-K\quad\Longrightarrow\quad \lambda_{\pm}=-s\pm ia$$

## Implementation notes

Integrate the method between minibatch gradient computation and the parameter update for one or more parameter blocks. Maintain the current gradient g_t and the previous gradient g_{t-1}. Normalize them to u_t and v_t, then construct a rank-two skew operator A_t = alpha times (u_t v_t-transpose minus v_t u_t-transpose). Use the update direction d_t = -g_t + A_t g_t and apply theta <- theta + eta d_t. Estimate a local stability margin every K steps using finite-difference probes: perturb theta in two random directions, measure the resulting gradient changes, and run a few Arnoldi iterations on the resulting Jacobian-vector product. If the estimated spectral radius of I plus eta J exceeds 0.98, reduce alpha and eta; otherwise increase alpha slowly up to a preset maximum. The paper supplies the symmetric/skew decomposition, covariance Lyapunov equation, and complex-eigenvalue mechanism. The implementation estimates J, C, and response empirically. For a response estimate, inject a small gradient perturbation at time t and record the parameter or loss response at later lags. First test on a two-layer MLP and MNIST against SGD, Adam, and momentum SGD, using alpha in {0, 0.1, 0.3, 0.6, 1.0}. Log loss, gradient autocorrelation, estimated spectral radius, and perturbation response. The prediction is that stronger alpha produces damped oscillations with envelope approximately exp(-s t) and period approximately 2 pi divided by absolute value of a. Instability should begin near rho(I + eta J) = 1, with the measured boundary within 20 percent of the estimated boundary. The fluctuation-response residual should increase with alpha even when the loss remains exponentially relaxing.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built the rank-two skew rotational optimizer and numerically verified the core mechanism. The toy sweep confirmed the predicted Euler stability boundary (predicted 1.2, measured 1.2), damped-envelope scaling, oscillation frequency, and Lyapunov covariance to near machine precision. However, the small MLP experiment showed no optimization win: rotational MSE was 0.03645 versus 0.01916 for momentum SGD and 0.02089 for Adam, with higher runtime.

### Mechanism check

- Verdict: Built the rank-two skew rotational optimizer and numerically verified the core mechanism. The toy sweep confirmed the predicted Euler stability boundary (predicted 1.2, measured 1.2), damped-envelope scaling, oscillation frequency, and Lyapunov covariance to near machine precision. However, the small MLP experiment showed no optimization win: rotational MSE was 0.03645 versus 0.01916 for momentum SGD and 0.02089 for Adam, with higher runtime.
- Confidence: 9/10
- Limitations: The finite-difference Jacobian/Arnoldi controller, MNIST comparison, empirical fluctuation-response residual, and real-network alpha sweep were not implemented. The neural test used one seed and a small synthetic regression task; the toy covariance check used isotropic noise, so it did not test alpha-dependent FDT residuals.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0394
- Paired wins: 1/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [REPORT.md](https://synthcore.org/code/555/REPORT.md)
- [bench_report.json](https://synthcore.org/code/555/bench_report.json)
- [bench_run.py](https://synthcore.org/code/555/bench_run.py)
- [experiment.py](https://synthcore.org/code/555/experiment.py)
- [report.md](https://synthcore.org/code/555/report.md)
- [report_bench_2026-08-31T174305.md](https://synthcore.org/code/555/report_bench_2026-08-31T174305.md)
- [results.json](https://synthcore.org/code/555/results.json)
- [Download all files as ZIP](https://synthcore.org/download/555)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.