A law of robustness for two-layer neural networks with arbitrary weights

What the math gives to ML

The paper establishes a distribution-dependent impossibility law: fitting noise with a narrow two-layer piecewise-linear network forces a large input Lipschitz constant, even when all weights are unbounded. The key transferable asset is that the relevant capacity is the number of distinct realized kink hyperplanes, not the nominal number of hidden units, and that function-space geometry can replace bounded parameter-space arguments. This suggests practical width-selection and regularization rules that explicitly trade off robustness, interpolation error, Lipschitz sensitivity, and effective kink count. The strongest near-term use is a ReLU training controller or architecture diagnostic rather than treating the lower bound as a conventional loss guarantee.