Math: Singular perturbation

Singularly Perturbed Hierarchical Training

Train the output layer on a fast timescale and the hidden feature layer on a slow timescale, so output coefficients first fit the components representable by the current features before hidden directions move. Use residual plateaus to detect when the fast subsystem has approximately equilibrated, then increase the hidden-layer learning rate to begin the next feature-learning stage.