Saddle-Node Branch Tracking for Training Control

1. Exact integration point: apply this to a training hyperparameter that can cause abrupt behavior, such as learning rate, weight-decay strength, dropout probability, MoE routing temperature, or attractive replica-consensus strength. Use a fixed probe set and define $q$ as one scalar measured after a short relaxation window: validation loss, representation norm, or mean pairwise replica distance. 2. Pseudocode: sweep $\mu$ in small increments; warm-start training at each value from the previous checkpoint; estimate $G(q,\mu)$ by taking one or several optimizer steps and measuring $(q_{t+1}-q_t)/\Delta t$; fit a local quadratic surface to $G$; flag a candidate fold when $G\approx0$ and the finite-difference derivative $\partial_qG$ changes sign or approaches zero. Then use predictor-corrector continuation: predict $(q,\mu)$ along the previous tangent, correct by solving $G=0$ and the arclength constraint, and choose the stable branch with lower validation loss. 3. Taken from the paper's mechanism: saddle-node conditions, branch disappearance, and square-root scaling near the fold. Estimated empirically: the reduced order parameter, drift function $G$, derivatives, and fold coordinates. 4. First cheap experiment: use a two-layer MLP on MNIST and sweep learning rate over a range that includes stable and divergent training, with five seeds per value. Compare ordinary linear warmup against fold-aware continuation. The predicted signature is a sharp minimum in $|\partial_qG|$ at the transition, two branches of $q$ separated by approximately $|\mu-\mu_c|^{1/2}$ on the existing side, and an abrupt jump when a naive sweep crosses $\mu_c$. A successful controller should avoid the jump while reaching the same target hyperparameter; estimate whether detected $\mu_c$ predicts the first loss spike within 20%.