Cycle-Aware Q-Order Scheduler

Integrate this into the optimizer scheduler, not into the forward network. The primary target is AdamW or SGD with momentum applied to a model with a known update period p, such as p=2 for alternating minimization or p=4 for a four-phase schedule. At every optimizer step, save a detached low-dimensional state z_k. To avoid storing all parameters, choose a fixed random projection R with r=16 to 64 columns and compute z_k=R transpose theta_k, or use the scalar validation loss. Maintain an exponential reference bar_z_k = beta bar_z_{k-1} + (1-beta) z_k, then compute e_k = norm(z_k-bar_z_k)_2 + epsilon. Store the most recent 2p errors. Once they exist, calculate c_hat_{p,k}=e_{k+p}/e_k and, when both ratios are positive and sufficiently far from one, calculate alpha_hat_k from the three p-spaced errors. Smooth c_hat with a median window W to reject minibatch noise. At cycle boundaries, reduce the learning rate by gamma_down=0.5 if the median factor exceeds tau_bad, increase it mildly by gamma_up=1.05 if it is below tau_good, and otherwise leave it unchanged; impose a cooldown of several cycles. The reference, projection dimension, thresholds, and smoothing window are empirical choices, while the p-spaced ratios are the mathematical quantities being transferred. First test a two-layer MLP and a small Transformer on MNIST or CIFAR-10 with SGD momentum and AdamW, deliberately choosing a learning rate that produces visible two-step oscillations. Compare fixed learning rates, ReduceLROnPlateau, and this scheduler for p=1, 2, and 4. Success means lower validation loss at equal optimizer steps, fewer instability spikes, and no premature learning-rate reduction when one-step loss oscillates but the p-step envelope decreases.