△ Mechanism confirmed, baseline not beaten 2026

FDT-Calibrated Rotational Optimizer

Implementation & benchmark of arXiv:2607.25782 — Fluctuation-dissipation violations in mean-field non-reciprocal spin glasses

Usefulness7/10
Difficulty6/10
Novelty7/10

Source paper: Fluctuation-dissipation violations in mean-field non-reciprocal spin glasses arXiv:2607.25782 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Add a controlled antisymmetric component to the local parameter update so optimization can circulate around ill-conditioned valleys instead of moving only along gradient directions. The symmetric component supplies dissipation, while the skew component produces the oscillatory non-reciprocal response predicted by the paper. Adapt the skew strength only while the estimated discrete-time dynamics remain stable.

Formulas

$$\dot{x}=(-S+A)x+\sqrt{2D}\,\xi(t),\qquad S=S^{T}\succeq 0,\quad A=-A^{T}$$
$$\rho(I+\eta J)<1,\qquad \max_k\operatorname{Re}\lambda_k(J)<0$$
$$JC+CJ^{T}+2D=0,\qquad R(t)=e^{Jt}$$
$$J=-sI+aK,\quad K^{T}=-K\quad\Longrightarrow\quad \lambda_{\pm}=-s\pm ia$$

Mathematical statement

Approximate a parameter block near its current state by the noisy linear dynamics d x/dt = J x + square-root(2D) xi(t), where x is a centered parameter displacement, J is the local Jacobian, D is gradient-noise covariance, and xi is white noise. Decompose J = -S + A, where S = S-transpose is the dissipative symmetric part and A = -A-transpose is the antisymmetric part. Continuous-time stability requires every eigenvalue of J to have negative real part; Euler updates require spectral radius rho(I + eta J) less than one, where eta is the learning rate. The stationary covariance C obeys J C + C J-transpose + 2D = 0. The impulse response is R(t) = exp(J t), while an equilibrium fluctuation-response relation would approximately connect R(t) to minus beta times the time derivative of C(t). Their mismatch measures broken detailed balance. For J = -s I + a K with K antisymmetric, eigenvalues are -s plus or minus i a, giving decay rate s and oscillation frequency absolute value of a.

Implementation notes

Integrate the method between minibatch gradient computation and the parameter update for one or more parameter blocks. Maintain the current gradient g_t and the previous gradient g_{t-1}. Normalize them to u_t and v_t, then construct a rank-two skew operator A_t = alpha times (u_t v_t-transpose minus v_t u_t-transpose). Use the update direction d_t = -g_t + A_t g_t and apply theta <- theta + eta d_t. Estimate a local stability margin every K steps using finite-difference probes: perturb theta in two random directions, measure the resulting gradient changes, and run a few Arnoldi iterations on the resulting Jacobian-vector product. If the estimated spectral radius of I plus eta J exceeds 0.98, reduce alpha and eta; otherwise increase alpha slowly up to a preset maximum. The paper supplies the symmetric/skew decomposition, covariance Lyapunov equation, and complex-eigenvalue mechanism. The implementation estimates J, C, and response empirically. For a response estimate, inject a small gradient perturbation at time t and record the parameter or loss response at later lags. First test on a two-layer MLP and MNIST against SGD, Adam, and momentum SGD, using alpha in {0, 0.1, 0.3, 0.6, 1.0}. Log loss, gradient autocorrelation, estimated spectral radius, and perturbation response. The prediction is that stronger alpha produces damped oscillations with envelope approximately exp(-s t) and period approximately 2 pi divided by absolute value of a. Instability should begin near rho(I + eta J) = 1, with the measured boundary within 20 percent of the estimated boundary. The fluctuation-response residual should increase with alpha even when the loss remains exponentially relaxing.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built the rank-two skew rotational optimizer and numerically verified the core mechanism. The toy sweep confirmed the predicted Euler stability boundary (predicted 1.2, measured 1.2), damped-envelope scaling, oscillation frequency, and Lyapunov covariance to near machine precision. However, the small MLP experiment showed no optimization win: rotational MSE was 0.03645 versus 0.01916 for momentum SGD and 0.02089 for Adam, with higher runtime.

Agent confidence
9/10
Baseline
Momentum SGD: final MSE 0.0191641, 0.928 s; Adam: final MSE 0.0208938, 1.239 s; SGD: final MSE 0.0407253, 1.157 s
Idea
Rotational optimizer: final MSE 0.0364512, 2.695 s; toy boundary a*=1.2 predicted and 1.2 measured; envelope and frequency predictions matched to approximately 1e-15

Limitations: The finite-difference Jacobian/Arnoldi controller, MNIST comparison, empirical fluctuation-response residual, and real-network alpha sweep were not implemented. The neural test used one seed and a small synthetic regression task; the toy covariance check used isotropic noise, so it did not test alpha-dependent FDT residuals.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.25782)

Significantly WORSE than the tuned baseline (+67.4%, p=0.0394, wins 1 / 8 paired seeds).

Benchmark
Tabular regression (Friedman)
Model
mlp_tiny
Paired seeds
8
Baseline mean
11.38
Idea mean
19.0459
Effect (Δ)
7.6659 (+67.4%; negative = idea better)
Wins
1 / 8 paired seeds
p-value
0.0394 (permutation test, 20 000 shuffles)
Smallest detectable effect
±84.1%
Mechanism
Confirmed ✓
Practical effect
Hurts
Baseline tuning
swept over 6 configs
Limitations:

Only the structurally matched tabular optimizer track was tested; vision, sequence, and dynamics tracks were not run. The implementation uses a finite-difference directional Jacobian signature rather than the planned Arnoldi spectral-radius controller, and the signature confirms orthogonality/skew magnitude but does not quantitatively establish complex eigenvalues or an empirical fluctuation-response residual. The baseline sweep uses the canonical bench Adam path; momentum SGD and a full optimizer-family comparison were not included in this stage-2 report.

How to run: python3 bench_run.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark latest Failed ✗

Agent confidence: 10/10

Built the FDT-Calibrated Rotational Optimizer as a local modification of the benchmark training loop and evaluated it on the structurally matched tabular optimizer track with the shared mlp_tiny architecture. Across 8 paired seeds, the best idea setting (lr=0.01, alpha=0.6) had test MSE 19.0459 versus 11.3800 for the tuned Adam baseline; paired delta was +7.6659 with permutation p=0.0394, meaning the idea was significantly worse. The trained-model mechanism signature confirmed the rank-two skew update was numerically orthogonal to the current gradient and had observed skew/gradient norm 0.6, but this did not yield a task-metric win.

Baseline
Tuned Adam, mlp_tiny, lr=0.01, weight_decay=0.0001: test MSE mean 11.3800330, std 1.0381924, per-seed [10.4924726, 12.0609503, 10.9885054, 10.6075563, 9.6461325, 11.8970881, 12.6752825, 12.6722765]
Idea
Rotational optimizer, mlp_tiny, lr=0.01, alpha=0.6: test MSE mean 19.0458884, std 11.0225469, per-seed [10.9603662, 18.7531452, 37.4108963, 12.7655478, 9.86921597, 14.0481577, 10.7555294, 37.8042488]; paired delta +7.6658554, p=0.0394

Benchmark result

Significantly WORSE than the tuned baseline (+67.4%, p=0.0394, wins 1 / 8 paired seeds).

Benchmark
Tabular regression (Friedman)
Model
mlp_tiny
Paired seeds
8
Baseline mean
11.38
Idea mean
19.0459
Effect (Δ)
7.6659 (+67.4%; negative = idea better)
Wins
1 / 8 paired seeds
p-value
0.0394 (permutation test, 20 000 shuffles)
Smallest detectable effect
±84.1%
Mechanism
Confirmed ✓
Practical effect
Hurts
Baseline tuning
swept over 6 configs
Limitations:

Only the structurally matched tabular optimizer track was tested; vision, sequence, and dynamics tracks were not run. The implementation uses a finite-difference directional Jacobian signature rather than the planned Arnoldi spectral-radius controller, and the signature confirms orthogonality/skew magnitude but does not quantitatively establish complex eigenvalues or an empirical fluctuation-response residual. The baseline sweep uses the canonical bench Adam path; momentum SGD and a full optimizer-family comparison were not included in this stage-2 report.

How to run: python3 bench_run.py

Artifacts

Implementation overview ⬇ Download all as ZIP 7 files · code, reports and structured results