Triangular Slow-Mode Optimizer

1. Integration point: implement AdamW or SGD with an additional optimizer state z_t having the same shape as the model parameters. Let x_t be the ordinary parameter update state. Use x_{t+1}=x_t-eta_g g_t+alpha z_t and z_{t+1}=rho z_t-eta_a h_t, where g_t is the current gradient and h_t is an exponential moving average of gradients. Detach h_t from the computation of z_t so that z influences x, while x does not feed back into the active-state evolution rule. This realizes an approximately triangular state transition. 2. Pseudocode: compute g=grad(L,x); update h=beta*h+(1-beta)*g; update z=rho*z-eta_a*h.detach(); update x=x-eta_g*g+alpha*z. Every K steps, estimate the sensitivity of z_{t+1} to x_t using a random-vector finite difference or a Jacobian-vector product, and add lambda_tri times its squared norm to a hyperparameter objective. Track A_Theta=norm(g) and A_J=norm(z), after projecting z orthogonally to g. 3. Taken from the paper: the triangular coupling structure and the prediction that the mode-rate gap is controlled by an anomalous exponent. Estimated empirically: eta, decay rates from regressions of log amplitudes against log training time, and the Jacobian blocks. 4. First experiment: train a two-layer MLP on MNIST with SGD, AdamW, and this optimizer using identical learning-rate grids. Initialize z so A_J/A_Theta=100 and sweep alpha. The falsifiable signature is a crossover satisfying log(t_cross) approximately equal to eta_inverse times log(A_J/A_Theta), with A_J/A_Theta decaying as t^{-eta}. Also test whether the measured lower-left Jacobian norm divided by the upper-right norm remains below 0.2 and whether the divergence threshold in alpha is higher than for an unconstrained bidirectional two-state optimizer.