FDT-Calibrated Rotational Optimizer
Implementation & benchmark of arXiv:2607.25782 — Fluctuation-dissipation violations in mean-field non-reciprocal spin glasses
Source paper: Fluctuation-dissipation violations in mean-field non-reciprocal spin glasses arXiv:2607.25782 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Add a controlled antisymmetric component to the local parameter update so optimization can circulate around ill-conditioned valleys instead of moving only along gradient directions. The symmetric component supplies dissipation, while the skew component produces the oscillatory non-reciprocal response predicted by the paper. Adapt the skew strength only while the estimated discrete-time dynamics remain stable.
Formulas
Mathematical statement
Approximate a parameter block near its current state by the noisy linear dynamics d x/dt = J x + square-root(2D) xi(t), where x is a centered parameter displacement, J is the local Jacobian, D is gradient-noise covariance, and xi is white noise. Decompose J = -S + A, where S = S-transpose is the dissipative symmetric part and A = -A-transpose is the antisymmetric part. Continuous-time stability requires every eigenvalue of J to have negative real part; Euler updates require spectral radius rho(I + eta J) less than one, where eta is the learning rate. The stationary covariance C obeys J C + C J-transpose + 2D = 0. The impulse response is R(t) = exp(J t), while an equilibrium fluctuation-response relation would approximately connect R(t) to minus beta times the time derivative of C(t). Their mismatch measures broken detailed balance. For J = -s I + a K with K antisymmetric, eigenvalues are -s plus or minus i a, giving decay rate s and oscillation frequency absolute value of a.
Implementation notes
Integrate the method between minibatch gradient computation and the parameter update for one or more parameter blocks. Maintain the current gradient g_t and the previous gradient g_{t-1}. Normalize them to u_t and v_t, then construct a rank-two skew operator A_t = alpha times (u_t v_t-transpose minus v_t u_t-transpose). Use the update direction d_t = -g_t + A_t g_t and apply theta <- theta + eta d_t. Estimate a local stability margin every K steps using finite-difference probes: perturb theta in two random directions, measure the resulting gradient changes, and run a few Arnoldi iterations on the resulting Jacobian-vector product. If the estimated spectral radius of I plus eta J exceeds 0.98, reduce alpha and eta; otherwise increase alpha slowly up to a preset maximum. The paper supplies the symmetric/skew decomposition, covariance Lyapunov equation, and complex-eigenvalue mechanism. The implementation estimates J, C, and response empirically. For a response estimate, inject a small gradient perturbation at time t and record the parameter or loss response at later lags. First test on a two-layer MLP and MNIST against SGD, Adam, and momentum SGD, using alpha in {0, 0.1, 0.3, 0.6, 1.0}. Log loss, gradient autocorrelation, estimated spectral radius, and perturbation response. The prediction is that stronger alpha produces damped oscillations with envelope approximately exp(-s t) and period approximately 2 pi divided by absolute value of a. Instability should begin near rho(I + eta J) = 1, with the measured boundary within 20 percent of the estimated boundary. The fluctuation-response residual should increase with alpha even when the loss remains exponentially relaxing.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Confirmed ✓
Stage 2 · Practical benchmark result: Significantly worse ✗
Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built the rank-two skew rotational optimizer and numerically verified the core mechanism. The toy sweep confirmed the predicted Euler stability boundary (predicted 1.2, measured 1.2), damped-envelope scaling, oscillation frequency, and Lyapunov covariance to near machine precision. However, the small MLP experiment showed no optimization win: rotational MSE was 0.03645 versus 0.01916 for momentum SGD and 0.02089 for Adam, with higher runtime.
- Agent confidence
- 9/10
- Baseline
- Momentum SGD: final MSE 0.0191641, 0.928 s; Adam: final MSE 0.0208938, 1.239 s; SGD: final MSE 0.0407253, 1.157 s
- Idea
- Rotational optimizer: final MSE 0.0364512, 2.695 s; toy boundary a*=1.2 predicted and 1.2 measured; envelope and frequency predictions matched to approximately 1e-15
Limitations: The finite-difference Jacobian/Arnoldi controller, MNIST comparison, empirical fluctuation-response residual, and real-network alpha sweep were not implemented. The neural test used one seed and a small synthetic regression task; the toy covariance check used isotropic noise, so it did not test alpha-dependent FDT residuals.
How to run: python3 experiment.py
Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.25782)
Significantly WORSE than the tuned baseline (+67.4%, p=0.0394, wins 1 / 8 paired seeds).
- Benchmark
- Tabular regression (Friedman)
- Model
- mlp_tiny
- Paired seeds
- 8
- Baseline mean
- 11.38
- Idea mean
- 19.0459
- Effect (Δ)
- 7.6659 (+67.4%; negative = idea better)
- Wins
- 1 / 8 paired seeds
- p-value
- 0.0394 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±84.1%
- Mechanism
- Confirmed ✓
- Practical effect
- Hurts
- Baseline tuning
- swept over 6 configs
Only the structurally matched tabular optimizer track was tested; vision, sequence, and dynamics tracks were not run. The implementation uses a finite-difference directional Jacobian signature rather than the planned Arnoldi spectral-radius controller, and the signature confirms orthogonality/skew magnitude but does not quantitatively establish complex eigenvalues or an empirical fluctuation-response residual. The baseline sweep uses the canonical bench Adam path; momentum SGD and a full optimizer-family comparison were not included in this stage-2 report.
How to run: python3 bench_run.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Stage 2 — Benchmark latest Failed ✗
Agent confidence: 10/10
Built the FDT-Calibrated Rotational Optimizer as a local modification of the benchmark training loop and evaluated it on the structurally matched tabular optimizer track with the shared mlp_tiny architecture. Across 8 paired seeds, the best idea setting (lr=0.01, alpha=0.6) had test MSE 19.0459 versus 11.3800 for the tuned Adam baseline; paired delta was +7.6659 with permutation p=0.0394, meaning the idea was significantly worse. The trained-model mechanism signature confirmed the rank-two skew update was numerically orthogonal to the current gradient and had observed skew/gradient norm 0.6, but this did not yield a task-metric win.
- Baseline
- Tuned Adam, mlp_tiny, lr=0.01, weight_decay=0.0001: test MSE mean 11.3800330, std 1.0381924, per-seed [10.4924726, 12.0609503, 10.9885054, 10.6075563, 9.6461325, 11.8970881, 12.6752825, 12.6722765]
- Idea
- Rotational optimizer, mlp_tiny, lr=0.01, alpha=0.6: test MSE mean 19.0458884, std 11.0225469, per-seed [10.9603662, 18.7531452, 37.4108963, 12.7655478, 9.86921597, 14.0481577, 10.7555294, 37.8042488]; paired delta +7.6658554, p=0.0394
Benchmark result
Significantly WORSE than the tuned baseline (+67.4%, p=0.0394, wins 1 / 8 paired seeds).
- Benchmark
- Tabular regression (Friedman)
- Model
- mlp_tiny
- Paired seeds
- 8
- Baseline mean
- 11.38
- Idea mean
- 19.0459
- Effect (Δ)
- 7.6659 (+67.4%; negative = idea better)
- Wins
- 1 / 8 paired seeds
- p-value
- 0.0394 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±84.1%
- Mechanism
- Confirmed ✓
- Practical effect
- Hurts
- Baseline tuning
- swept over 6 configs
Only the structurally matched tabular optimizer track was tested; vision, sequence, and dynamics tracks were not run. The implementation uses a finite-difference directional Jacobian signature rather than the planned Arnoldi spectral-radius controller, and the signature confirms orthogonality/skew magnitude but does not quantitatively establish complex eigenvalues or an empirical fluctuation-response residual. The baseline sweep uses the canonical bench Adam path; momentum SGD and a full optimizer-family comparison were not included in this stage-2 report.
How to run: python3 bench_run.py
Artifacts
- REPORT.md 3.0 KB View
- bench_report.json 3.9 KB View Raw
- bench_run.py 6.6 KB View Raw
- experiment.py 6.9 KB View Raw
- report.md 1.4 KB View
- report_bench_2026-08-31T174305.md 6.4 KB View
- results.json 4.7 KB View Raw