FDT-Calibrated Rotational Optimizer MVP

Implementation

experiment.py implements the rank-two skew update

A = alpha (u v^T - v u^T), followed by theta <- theta + eta(-g + A g).

It also contains a conservative alpha controller, a linear two-dimensional dynamics sweep, Lyapunov covariance verification, and a small CUDA/CPU-safe MLP regression comparison against SGD, momentum SGD, and Adam.

Toy mechanism verification

Parameters were s=0.4, eta=0.5, and D=0.7, with J=-sI+aK.

1. Discrete stability boundary. The prediction is rho(I+eta J)=sqrt((1-eta*s)^2+(eta*a)^2) and instability begins at a*=sqrt(2s/eta-s^2)=1.2. Independent bisection on the actual update matrix measured a*=1.1999999999999997, relative error 1.85e-16. The sweep was stable at a=1.1571 (rho=0.98729) and unstable at a=1.3886 (rho=1.05926), correctly bracketing the transition.

2. Damped envelope. The predicted per-step log envelope is log(rho). Across all eight values of a from 0 to 1.62, the fitted trajectory slope matched the prediction to approximately 1e-15; examples: at a=0.6943, observed -0.1368842166 versus predicted -0.1368842166, and at a=1.3886, observed 0.0575709546 versus predicted 0.0575709546.

3. Oscillation frequency. The predicted discrete angular frequency is atan2(eta*a, 1-eta*s). The measured phase slope matched at every sweep point; examples: a=0.4629, observed/predicted 0.2815984218, and a=1.1571, observed/predicted 0.6261367182 radians per step. Frequency is zero when a=0 and increases with skew strength.

4. Stationary covariance. Solving the Lyapunov equation numerically produced C=(D/s)I=1.75I for a=0, 0.3, 0.8; off-diagonal values were below 1.3e-16. This confirms the isotropic-noise prediction that rotation changes response/trajectory but not the stationary covariance in this special case.

These checks confirm the claimed antisymmetric complex-eigenvalue mechanism and its discrete stability condition.

Mini-experiment

Fixed-seed 500-step, batch-64 training of a 12-32-1 tanh MLP on a synthetic regression task, run on CUDA. Final full-dataset MSE:

| Optimizer | MSE | seconds | |---|---:|---:| | SGD | 0.0407253 | 1.157 | | Momentum SGD | 0.0191641 | 0.928 | | Adam | 0.0208938 | 1.239 | | Rotational optimizer | 0.0364512 | 2.695 |

The proposed optimizer was worse than momentum SGD and Adam and took about 2.3x the SGD runtime in this implementation. Thus the mechanism is real, but no practical optimization win was observed in this tiny test.

Reproduction

/home/maxwelhelp/main/bin/python3 experiment.py

Results are written to results.json.

Limitations

The requested finite-difference Jacobian/Arnoldi stability estimator, MNIST experiment, empirical fluctuation-response perturbation residual, and alpha sweep on a real network were not implemented. The toy covariance test uses isotropic noise and therefore cannot demonstrate alpha-dependent FDT residuals. The MLP comparison uses one seed and synthetic regression rather than MNIST.