Approximation of Analytic Functions by ReLU Neural Networks with Adjustable Depth and Width
arXiv:2607.10589
2026
Architecture
2 ideas extracted · analyzed Aug 30, 2026
What the math gives to ML
The paper gives constructive approximation results showing that ReLU networks can approximate analytic functions with error decreasing exponentially in a depth parameter, roughly as N^{-C L}, rather than only algebraically in width and depth. The transferable asset is the explicit construction of approximate powers, Legendre polynomials, multiplication, and multivariate polynomial expansions using ReLU subnetworks. This suggests allocating a fixed parameter budget toward depth for smooth coordinate-based regression, neural fields, and scientific machine-learning problems. The most concrete adaptation is a polynomial-feature trunk whose approximation error can be measured directly on a dense input grid.
Ideas from this paper
Unverified
2026
Construct a reusable ReLU trunk that emits approximate univariate powers or Legendre-polynomial features for each input coordinate, then combine them with a linear or low-rank polynomial head. This gives a compact explicit basis for smooth functions and can replace a large generic MLP in low-dimensional scientific regression or serve as a frozen or partially trainable front-end.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
For smooth coordinate-based regression, replace a width-heavy MLP with a deliberately narrow but deeper ReLU network and choose depth and width using the paper's analytic-function approximation law. The hypothesis is that, at fixed parameter count, increasing depth gives a larger reduction in approximation error than increasing width when the target is close to analytic.
Useful6/10
Difficulty4/10
Novelty7/10