Spectral Convergence of Random Feature Method in Multiple Dimensions
arXiv:2609.03401
2026
Architecture
2 ideas extracted · analyzed Sep 4, 2026
What the math gives to ML
The paper provides a principled method for choosing random Fourier feature distributions from the expected regularity of the target, rather than using a fixed Gaussian or uniform frequency law. Its transferable asset is spectral approximation: Sobolev targets obtain algebraic rates, Gevrey targets stretched-exponential rates, and ultra-analytic or bandlimited targets super-exponential rates as the feature count grows. This can become a frozen, regularity-adapted input layer followed by a trainable linear or shallow nonlinear head. Because the paper also identifies severe ill-conditioning as the price of spectral accuracy, implementations should combine the feature distribution with whitening or ridge stabilization.
Ideas from this paper
Unverified
2026
Replace the usual isotropic Gaussian random Fourier features with a frequency distribution matched to the expected spectral regularity of the target function. For coordinate fields, operator-learning maps, or PDE solution surrogates, this should place more features where the target Fourier energy lies and improve approximation at the same feature count. Stabilize the resulting feature matrix with whitening or ridge regression because spectral accuracy can create severe ill-conditioning.
Useful7/10
Difficulty4/10
Novelty5/10
Unverified
2026
For bandlimited or progressively higher-frequency targets, construct a nested random Fourier layer whose frequency window grows according to the paper's bandwidth laws instead of sampling all frequencies from one fixed range. Train with a low-bandwidth subset first, then activate additional frequency blocks. This creates a mathematically motivated spectral curriculum and may reduce early optimization difficulty while preserving high-frequency accuracy.
Useful6/10
Difficulty5/10
Novelty6/10