Slepian-Concentration Feature Initialization
Implementation & benchmark of arXiv:2607.19192 — Optimal concentration in the Paley-Wiener space
Source paper: Optimal concentration in the Paley-Wiener space arXiv:2607.19192 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Initialize a coordinate-network feature bank with the leading eigenfunctions of a bandlimited concentration operator instead of random Fourier features. For a desired spatial region E, these features maximize the fraction of their L2 energy inside E among all functions with frequency support in Omega, giving a principled basis for localized signals.
Formulas
Mathematical statement
The paper defines the Paley-Wiener space PW(Omega)={f in L2(R): supp fhat is contained in Omega}, where fhat(xi)=integral over R of f(x) exp(-2 pi i x xi) dx. The orthogonal projection P_Omega has reproducing kernel k(x-y)=integral over Omega of exp(2 pi i xi(x-y)) d xi; for Omega=[-1/2,1/2], this is k(x-y)=sin(pi(x-y))/(pi(x-y)), with value 1 at x=y. Let M_E f=1_E f. The concentration is the Rayleigh quotient lambda(E)=sup over nonzero f in PW(Omega) of ||1_E f||_2^2/||f||_2^2, equivalently the largest eigenvalue of C_E=P_Omega M_E P_Omega. The paper proves that if E_star is an interval with |E_star|=|E|, then lambda(E)<=lambda(E_star). We use the leading eigenfunctions of a discretized C_E_star as localized bandlimited neural features.
Implementation notes
(1) Integration point: use this as a fixed feature map in a one-dimensional coordinate MLP or as initialization for a localized convolutional or implicit-neural-representation basis. Normalize coordinates to [-L/2,L/2], choose a target interval E_star=[-T/2,T/2], and choose a frequency bandwidth Omega=[-B/2,B/2]. The input feature tensor is Phi(x)=[phi_1(x),...,phi_r(x)], optionally concatenated with x.
(2) Pseudocode: sample grid points x_i with spacing delta_x. Form K_ij=delta_x k_B(x_i-x_j), where k_B(u)=B sin(pi B u)/(pi B u), using k_B(0)=B. Form the diagonal mask D_E with entries 1 when x_i belongs to E_star and 0 otherwise. Compute the symmetric concentration matrix C=K D_E K, then compute its top r eigenvectors v_j with Lanczos iteration. Interpolate each v_j over x using cubic splines and feed Phi(x) into the MLP. The discrete eigenproblem approximates C_E phi_j=lambda_j phi_j. Freeze the basis initially; optionally learn a small linear mixing matrix W after Phi.
(3) Computed from the paper's mathematics: the sinc projection kernel, concentration matrix, leading eigenvectors, and eigenvalues. Estimated empirically: bandwidth B, interval width T, feature count r, and whether fine-tuning improves results. Normalize each eigenvector to unit discrete L2 norm after interpolation.
(4) First cheap experiment: fit one-dimensional signals made from Gaussian bumps and piecewise-smooth segments with a coordinate MLP. Compare concentration features, random Fourier features, and sinusoidal positional encoding using equal feature count, hidden width, and training FLOPs. Measure validation MSE versus labeled-sample count, optimization loss versus step count, and the fraction of prediction energy lying in E_star. The expected signal is lower error and faster loss descent for targets concentrated in E_star. Use a fragmented set with the same measure as a control; the theorem predicts that an interval is the optimal concentration baseline, not that it is optimal for every target distribution.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.