Cascading Through the Hierarchy: Regularizer-Induced Feature Detection as Phase Transitions in Deep Linear Neural Networks
The paper provides a tractable mechanism by which increasing L2 regularization causes a cascade of feature-detection phase transitions in deep linear networks. Under whitened inputs and approximately balanced, singular-vector-aligned…