Endpoint-transformed Hermite feature layer
Implementation & benchmark of arXiv:2607.12648 — Hermite spectral approximation for functions with endpoint singularities using exponential transforms
Source paper: Hermite spectral approximation for functions with endpoint singularities using exponential transforms arXiv:2607.12648 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace ordinary Fourier, polynomial, or raw-coordinate features for a bounded scalar coordinate with Hermite functions evaluated after a monotone endpoint transform. The transform sends endpoint singularities to localized tails on the real line, while a learnable scale controls how many Hermite modes are needed. This is suited to coordinate MLPs, neural operators, and implicit fields whose targets have square-root, logarithmic, boundary-layer, or derivative singularities.
Formulas
Mathematical statement
The paper defines normalized Hermite functions $\psi_n(x)=\frac{(-1)^n}{\sqrt{\gamma_n}}e^{x^2/2}\frac{d^n}{dx^n}e^{-x^2}$ with $\gamma_n=2^n n!\sqrt{\pi}$, and they satisfy orthonormality $\int_{\mathbb R}\psi_j(x)\psi_k(x)\,dx=\delta_{j,k}$. Let $u\in(0,1)$ be a bounded network coordinate, let $\varphi:(0,1)\to\mathbb R$ be a strictly increasing SE, DE, or EF transform with divergent tails at the endpoints, and let $\lambda>0$ be a scale. Use the transformed basis $\Psi_{k,\lambda}(u)=\sqrt{\lambda}\,\psi_k(\lambda\varphi(u))$ and model $\hat f(u)=\sum_{k=0}^{K-1}a_k\Psi_{k,\lambda}(u)$. The derivative identity $\psi^{\prime}_{n}(x)=\sqrt{n/2}\psi_{n-1}(x)-\sqrt{(n+1)/2}\psi_{n+1}(x)$ yields $\frac{d\hat f}{du}=\lambda\varphi'(u)\sum_{k=0}^{K}b_k\Psi_{k,\lambda}(u)$, where $b_k=\sqrt{(k+1)/2}a_{k+1}-\sqrt{k/2}a_{k-1}$ and $a_K=a_{K+1}=0$. The assumptions are that the transformed target is square-integrable and sufficiently smooth and decaying; the paper claims root-exponential convergence without scale tuning and substantially improved convergence with suitable scaling.
Implementation notes
Integrate the method at the input embedding of a coordinate MLP. For each bounded scalar coordinate $u\in(0,1)$, compute $z=\lambda\varphi(u)$ and concatenate $[\Psi_{0,\lambda}(u),\ldots,\Psi_{K-1,\lambda}(u)]$ to the ordinary MLP input. Clamp $u$ to $[\epsilon,1-\epsilon]$ and clamp $z$ to a safe range before evaluating the basis. Compute Hermite values using the stable recurrence psi[0]=pi**(-.25)*exp(-.5*z*z), psi[1]=sqrt(2)*z*psi[0], and psi[k+1]=sqrt(2/(k+1))*z*psi[k]-sqrt(k/(k+1))*psi[k-1]. Then set features=sqrt(lambda)*psi and evaluate y=MLP(concat(raw_input,features)). Initialize log_lambda from a logarithmic sweep and optionally learn it jointly under bounds. For derivative supervision, use the paper's sparse coefficient rule: compute b[k]=sqrt((k+1)/2)*a[k+1]-sqrt(k/2)*a[k-1], multiply the resulting basis sum by lambda*phi_prime(u), and avoid differentiating each feature through a long automatic-differentiation graph. The mathematical pieces are the orthonormal basis, monotone transform, scale, and sparse derivative recurrence; transform clipping, scale bounds, and hidden-layer width are empirical choices. First test a 1D coordinate MLP on 1,000 samples of $f(u)=\sqrt{u}+\log(1+u)+\sqrt{1-u}$, comparing raw coordinates, Fourier features, Legendre features, and transformed Hermite features at equal parameter count. Sweep $K\in\{8,16,32\}$ and fixed versus learned $\lambda$. Success means lower endpoint-inclusive test error and a steeper error-versus-$K$ curve; a concrete target is 2x lower error than Fourier features at $K=16$ or matching baseline error with half the hidden parameters.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.