Automated Numerical Stability Analysis of Deep Learning Operators

What the math gives to ML

The paper models neural computation as a typed dependency graph and attaches stochastic-arithmetic diagnostics to individual operators rather than only inspecting the final loss. This enables numerical instability to be localized to a specific GEMM, normalization, reduction, or activation during training or inference. The transferable asset is an online operator-level perturbation score that can trigger selective higher-precision recomputation or a numerically safer implementation only where instability is detected, preserving low precision elsewhere.