Bounded-Extrapolation Fourier Layer
Implementation & benchmark of arXiv:2607.10501 — Optimal Extrapolation Bounds for Sparse Fourier Sums
Source paper: Optimal Extrapolation Bounds for Sparse Fourier Sums arXiv:2607.10501 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace an unconstrained Fourier-feature block in an implicit neural representation or coordinate MLP with a sparsity-aware layer whose output gain is normalized according to the distance outside the training interval. The normalization uses the paper's endpoint law, preventing a small in-domain Fourier representation from producing arbitrarily large outputs just beyond the observed coordinate range.
Formulas
Mathematical statement
For a scalar k-sparse Fourier sum g(t)=sum_{j=1}^k v_j exp(i lambda_j t), where lambda_j are real frequencies and v_j are complex coefficients, the paper bounds extrapolation from [-1,1] by |g(x)| <= k^{O(1)} exp(O(k arcosh(x))) ||g||_{L^2[-1,1]} for x >= 1. Near the endpoint, for x=1+delta and 0 <= delta <= 1, it gives |g(1+delta)| <= O(k) exp(O(k sqrt(delta))) ||g||_{L^2[-1,1]}. The norm is ||g||_{L^2[-1,1]}=(integral_{-1}^1 |g(t)|^2 dt)^{1/2}; delta is the normalized distance outside the observed interval; and k is the number of active atoms. Use a calibrated envelope B(k,delta)=C_0 k^p exp(C_1 k sqrt(delta)) near the endpoint, or B(k,x)=C_0 k^p exp(C_1 k arcosh(x)) farther away.
Implementation notes
(1) Integration point: use this as the final operation of a Fourier-feature coordinate encoder, before concatenating its output with an MLP or before the first linear layer. Normalize training coordinates to [-1,1]. For a query t, define delta(t)=max(0,abs(t)-1), using the endpoint formula when delta <= 1 and the arcosh formula for larger coordinates. Apply the operation independently to each real or complex Fourier channel, or to the complete scalar Fourier head if channelwise normalization is too restrictive. (2) Pseudocode: sample in-domain coordinates t_n; compute z(t)=sum over active j of v_j exp(i lambda_j t); compute E_in=sqrt(mean_n abs(z(t_n))^2); set B=C0*k^p*exp(C1*k*sqrt(delta)) for delta<=1 and B=C0*k^p*exp(C1*k*acosh(abs(t))) otherwise; set z_safe=z*min(1,B*E_in/(abs(z)+eps)); pass real(z_safe) and imag(z_safe) downstream. (3) The theorem supplies the dependence on k and distance; C0,C1,p are hidden constants and must be estimated by maximizing log(abs(z(t))/(E_in+eps)) over random coefficients, frequencies, and boundary queries, then adding a safety margin. (4) First experiment: train SIREN and Fourier-feature coordinate MLPs on 1D signals and 2D images using only coordinates in [-0.8,0.8]. Compare an ordinary Fourier layer, hard clipping, and this layer at equal parameter count. Measure central reconstruction, adjacent extrapolation MSE, maximum absolute output, and gradient norm for queries 10% and 25% outside the interval. Success means lower boundary overshoot and extrapolation MSE without more than 0.2 dB in-domain PSNR loss.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.