High Probability Derivative Bounds for Random tanh Neural Networks on a Hypercube

What the math gives to ML

The paper replaces worst-case layer-norm multiplication with high-probability derivative control for wide random tanh networks. Under Xavier Gaussian initialization and an explicit width condition, first-order input sensitivity is independent of depth, while square-free mixed derivatives grow polynomially in depth rather than exponentially. The most direct neural-network transfer is a derivative regularizer whose order-dependent targets follow this scaling, combined with width-aware initialization and monitoring of Jacobian growth. This is particularly suitable for smooth surrogate models, robustness-sensitive regressors, and neural operators where input regularity matters.