Second-Order Deficit Update Scheduler

1. Integration point: partition a neural network into N update groups, such as transformer layers, LoRA adapters, task-specific heads, MoE experts, or data sources in a multi-task loader. At each optimizer step, permit only K groups to receive gradients, or choose K groups whose examples are admitted into the next microbatch. Initialize mu_i_star proportionally to estimated group importance, such as normalized gradient norm or validation-loss contribution, while enforcing sum_i mu_i_star = K. Set v_i_star to a small value to discourage bursts. 2. Pseudocode: compute w_i = alpha_i times max(mu_i_star - mu_hat_i, 0) plus gamma_i times abs(v_i_star - v_hat_i); select the top K groups; run forward and backward only for their associated parameters or samples; set x_i to one for selected groups and zero otherwise; update mu_hat_i and v_hat_i using the exponential equations above; then apply the optimizer update. 3. Computed from the mechanism: deficit scores, top-K decisions, and online first- and second-moment estimates. Estimated empirically: feasible target means, gradient-noise scales, and useful values of alpha_i, gamma_i, and beta. 4. First cheap experiment: train a small six-layer transformer on WikiText-2 using selective layer updates with K=3, comparing uniform round-robin, loss-proportional scheduling, and this controller at equal FLOPs. Use mu_i_star = K/N and v_i_star = 0.02 initially. Measure update-rate error, temporal variance, loss, and gradient-cosine oscillation. The falsifiable prediction is that, when sum_i mu_i_star <= K, mean update deficits decay toward zero and update-rate error is at least two times lower than with loss-proportional scheduling. Setting sum_i mu_i_star > K must create a persistent nonzero deficit, exposing the predicted feasibility boundary.