Slow-Mode-Canceling Optimizer Packet

1. Integration point: add a second trainable parameter vector phi with the same architecture as the ordinary model parameter vector theta. At every minibatch, compute both losses and update both replicas with gradients plus a parameter-coupling term. Use the target prediction from theta; use phi only as the packet and optionally average predictions late in training. The discrete Euler updates are theta <- theta-eta[g_theta+kappa(theta-phi)] and phi <- phi-eta[g_phi+kappa(phi-theta)]. 2. Pseudocode: initialize theta and phi, estimate a low-curvature direction v1 using 5--20 Hessian-vector products and Lanczos or inverse iteration, then estimate the displacement x relative to a nearby reference theta_star or an initial checkpoint. Set phi so that v1^T(phi-theta_star)=-v1^T(theta-theta_star), while leaving orthogonal components unchanged. Run the coupled updates and re-estimate v1 every 100--1000 steps. 3. Computed versus estimated: the modal equations and cancellation condition are analytic; v1, lambda_1, and lambda_2 are estimated from Hessian-vector products or by fitting exponential decay rates of projected errors. Minibatch noise will continually repopulate the cancelled mode, so measure its replenishment rate. 4. First cheap experiment: use a two-layer MLP on MNIST and compare SGD, an unprepared two-replica optimizer, and the prepared optimizer at matched total parameter-update FLOPs. Measure loss, validation accuracy, and error projected onto v1. The falsifiable prediction is that the late-time projected error changes from A exp(-lambda_1 t) to approximately B exp(-min(lambda_2,lambda_1+2kappa)t). Increasing kappa should improve the decay rate only until lambda_1+2kappa approximately equals lambda_2. A cancellation ratio |c_1(0)|/|a_1(0)| below 0.1 should yield at least a tenfold reduction in slow-mode amplitude. For discrete updates, instability should appear when eta(lambda_max+2kappa) reaches approximately 2.