Critical-Rate Learning-Rate Controller

1. Integration point: use this as an outer controller for the learning-rate or optimizer-gain schedule. At every optimizer step, retain a scalar observable x_hat measuring the potentially unstable training mode. A cheap choice is the projection of the parameter update onto the previous update, x_hat_t = clip((Delta theta_t dot Delta theta_{t-1})/(||Delta theta_{t-1}||^2+epsilon), -M, M); alternatively use the largest Ritz eigenvalue of the minibatch Hessian or the norm of a recurrent hidden state. Let u_t be a normalized schedule variable controlling learning rate eta_t = eta_min + sigmoid(u_t)(eta_max-eta_min). 2. Pseudocode: initialize u and an exponential moving estimate r_hat. For each step, compute the normal gradient update with eta(u); estimate x_hat and its one-step drift v_hat = (x_hat_t-x_hat_{t-1})/Delta_t; estimate local distance d_hat from the tracked branch, or use d_hat = sqrt(max(-u_t, epsilon)); compute r_prior = 4*max(-u_t, epsilon)^(3/2). Set r_hat = alpha*r_hat + (1-alpha)*abs(v_hat). If abs(delta_u/Delta_t) > gamma*r_prior or x_hat approaches the estimated unstable side, replace delta_u by sign(delta_u)*gamma*r_prior*Delta_t; otherwise keep the proposed schedule update. 3. The saddle-node equilibria, eigenvalue, and 4(-u)^(3/2) scaling are analytic. The mapping from network statistics to x_hat and the exact critical-rate function are empirical and should be estimated by short perturbation rollouts: temporarily increase the schedule rate by factors 1.0, 1.25, 1.5, and record whether x_hat returns to the tracked branch or overshoots. 4. First cheap experiment: train a small two-layer MLP on MNIST or CIFAR-10 with SGD, cosine warmup, and the controller versus the same schedule without control. Sweep warmup slopes over a logarithmic grid. Plot update norm, loss, x_hat, and schedule slope. The quantitative prediction is a sharp overshoot boundary: after normalizing by r_prior, runs with slope ratio below approximately 1 should remain tracking, while ratios above 1 should show a rapid increase in update norm or loss variance; the measured transition should fall within 20% of the predicted ratio. A secondary prediction is that adaptive control reduces overshoot events without reducing the maximum stable average learning rate.