Multiplier-Aware Anti-Flip Optimizer

1. Integration point: wrap SGD, momentum, Adam, or a recurrent-state update with a controller that runs every K optimizer steps. Begin with plain SGD so optimizer memory does not obscure the discrete-time mechanism. The monitored state can be all parameters for a small model or a low-dimensional random projection for a larger model. 2. Pseudocode: save a normalized probe vector v, evaluate one optimizer map F_eta(theta), and estimate its Jacobian action by finite difference: Jv approximately equals [F_eta(theta+epsilon v)-F_eta(theta)]/epsilon. Use two or three probes and several power-iteration steps. Compute the signed Rayleigh quotient r=v-transpose times Jv divided by v-transpose times v. If the smallest observed quotient is below -tau, set eta to eta times (1-gamma), and optionally interpolate the new update with the previous update. If all estimated multipliers remain inside (-0.8,0.8), restore eta slowly using eta times (1+gamma/4). 3. Computed from the paper's mechanism: the stability condition rho(J)<1, the -1 flip boundary, and geometric decay or growth. Estimated empirically: Jacobian-vector products, the local multiplier, and the approximate fixed point, because minibatch noise prevents an exact fixed point. 4. First cheap experiment: train a two-layer MLP on MNIST with plain SGD over a learning-rate grid, comparing fixed learning rate with the controller. Record even-odd loss differences, consecutive-update cosine similarity, loss variance, and the estimated multiplier. The prediction is that oscillatory training begins when the measured multiplier crosses -1 plus or minus 0.2, that even-odd update correlation becomes strongly negative before divergence, and that controlled runs maintain absolute multiplier below 0.9 with geometric suppression of perturbations.