Radon Measure Representations for Infinite-Width Neural Networks with Singular Activations

What the math gives to ML

The paper supplies a distributional way to remove the zero-frequency singularity that prevents direct Fourier/Radon analysis of common activations such as ReLU. Its transferable asset is the explicit spectral regularization \(\widehat{g}(\rho)=(i\rho)^\alpha\widehat{\sigma}(\rho)\), which converts a singular activation into a decay-compatible feature while remaining in ordinary tempered-distribution spaces rather than quotient spaces. A practical neural-network use is to build a spectrally preconditioned shallow feature map and test whether its induced design matrix is better conditioned than the original activation's feature map. The result is most naturally applied to random-feature MLPs or the first layer of a larger network, with a residual raw-activation path preserving approximation capacity.