Capacity-Triggered Hybrid Optimizer

Integrate the controller at every optimizer step, after computing the minibatch gradient and before applying the parameter update. Maintain an EMA load q, a regime variable r, and the optimizer state v. Use hysteresis to avoid rapid switching: enter regime 1 when q exceeds alpha*C and return to regime 0 only when q falls below (alpha-h)*C; enter regime 2 at C and leave it only below (1-h)*C. Pseudocode is: compute minibatch gradient g; compute phi as norm(g), update norm, or an occasional Hessian-vector-product curvature proxy; update q=(1-rho)*q+rho*phi; select r using the hysteresis thresholds; select eta_r, mu_r, beta_r; if r=2 replace g by g*min(1,C_g/(||g||_2+epsilon)) and optionally skip or accumulate the update; update v=mu_r*v+(1-mu_r)*g; update theta=theta-eta_r*(v+beta_r*g). The paper supplies the hybrid-regime and bifurcation mechanism, while C, alpha, h, rho, and controller gains must be calibrated empirically. First experiment: train a three-layer MLP on MNIST and a small Transformer on CIFAR-10, comparing AdamW with this controller at matched compute and matched initial learning rate. Sweep alpha, damping, and feedback strength. Record loss variance, gradient norm, q, regime occupancy, and update rejection rate. The quantitative prediction is a sharp stability boundary: as feedback delay or effective learning rate increases, oscillations should first appear near the estimated Jacobian eigenvalue crossing, with period approximately 2*pi/omega. The measured boundary should shift toward smaller learning rates or larger damping as controller gain increases, and training curves should show a kink when alpha crosses the value at which the stable trajectory first reaches the switching surface.