Subcritical Gradient-Cascade Control

1. Integration point: use a residual MLP, small Transformer, or GRU and apply the controller to optimizer updates. After computing minibatch gradient g at step m, form a perturbation with the same scale as the proposed update, delta_theta = -eta times g, or inject a small perturbation into one hidden unit or token. Run paired perturbed and unperturbed forward passes. Define an event as a unit-token-layer state for which the normalized difference exceeds epsilon: absolute(delta h) divided by activation standard deviation plus a small constant is greater than epsilon. 2. Pseudocode: sample B seed events; create generation-zero masks; propagate perturbed and reference states through layers or recurrent steps; threshold differences; remove events already counted; record n_t; estimate R_hat from the ratio formula; perform the normal optimizer update when R_hat is below R_safe, otherwise replace eta by the exponential controller formula. Measure every 50 to 200 optimizer steps to limit overhead. 3. Taken from the paper: the branching-process cascade law and the subcritical condition R less than 1. Estimated empirically: event masks, R_hat, cascade sizes, and the dependence of R_hat on eta. 4. First experiment: train a four-layer residual MLP on MNIST or a two-layer Transformer on Tiny Shakespeare using Adam, fixed-learning-rate baselines, and the controller. Sweep learning rates and record R_hat, cascade size, gradient norm, and loss divergence. The prediction is that mean cascade size follows 1/(1-R_hat) while R_hat is below 0.9, and that exploding gradients or unstable loss begins near R_hat equal to 1, within approximately 20 percent across widths.