# Endpoint-transformed Hermite feature layer

- ID: 821
- Canonical URL: https://synthcore.org/idea/821/endpoint-transformed-hermite-feature-layer
- API JSON: https://synthcore.org/api/idea/821.json
- API Markdown: https://synthcore.org/api/idea/821.md
- Verification status: unverified
- Source: [arXiv:2607.12648](https://arxiv.org/abs/2607.12648)
- Category: architecture
- Solves: accuracy, sample-efficiency, scalability
- ML areas: mlp, embedding, loss, initialization
- Math tags: approximation-theory, harmonic-analysis, linear-algebra, numerical-analysis
- Ratings: usefulness 6/10; difficulty 4/10; novelty 7/10

## Idea description

Replace ordinary Fourier, polynomial, or raw-coordinate features for a bounded scalar coordinate with Hermite functions evaluated after a monotone endpoint transform. The transform sends endpoint singularities to localized tails on the real line, while a learnable scale controls how many Hermite modes are needed. This is suited to coordinate MLPs, neural operators, and implicit fields whose targets have square-root, logarithmic, boundary-layer, or derivative singularities.

## Mathematical statement

The paper defines normalized Hermite functions $\psi_n(x)=\frac{(-1)^n}{\sqrt{\gamma_n}}e^{x^2/2}\frac{d^n}{dx^n}e^{-x^2}$ with $\gamma_n=2^n n!\sqrt{\pi}$, and they satisfy orthonormality $\int_{\mathbb R}\psi_j(x)\psi_k(x)\,dx=\delta_{j,k}$. Let $u\in(0,1)$ be a bounded network coordinate, let $\varphi:(0,1)\to\mathbb R$ be a strictly increasing SE, DE, or EF transform with divergent tails at the endpoints, and let $\lambda>0$ be a scale. Use the transformed basis $\Psi_{k,\lambda}(u)=\sqrt{\lambda}\,\psi_k(\lambda\varphi(u))$ and model $\hat f(u)=\sum_{k=0}^{K-1}a_k\Psi_{k,\lambda}(u)$. The derivative identity $\psi^{\prime}_{n}(x)=\sqrt{n/2}\psi_{n-1}(x)-\sqrt{(n+1)/2}\psi_{n+1}(x)$ yields $\frac{d\hat f}{du}=\lambda\varphi'(u)\sum_{k=0}^{K}b_k\Psi_{k,\lambda}(u)$, where $b_k=\sqrt{(k+1)/2}a_{k+1}-\sqrt{k/2}a_{k-1}$ and $a_K=a_{K+1}=0$. The assumptions are that the transformed target is square-integrable and sufficiently smooth and decaying; the paper claims root-exponential convergence without scale tuning and substantially improved convergence with suitable scaling.

## Key formulas

- $$\psi_n(x)=\frac{(-1)^n}{\sqrt{\gamma_n}}e^{x^2/2}\frac{d^n}{dx^n}e^{-x^2},\qquad \gamma_n=2^n n!\sqrt{\pi}.$$
- $$\int_{\mathbb R}\psi_j(x)\psi_k(x)\,dx=\delta_{j,k}.$$
- $$\hat f(u)=\sum_{k=0}^{K-1}a_k\Psi_{k,\lambda}(u),\qquad \Psi_{k,\lambda}(u)=\sqrt{\lambda}\,\psi_k\!\left(\lambda\varphi(u)\right).$$
- $$\frac{d\hat f}{du}=\lambda\varphi'(u)\sum_{k=0}^{K}b_k\Psi_{k,\lambda}(u),\qquad b_k=\sqrt{\frac{k+1}{2}}a_{k+1}-\sqrt{\frac{k}{2}}a_{k-1},\quad a_K=a_{K+1}=0.$$

## Implementation notes

Integrate the method at the input embedding of a coordinate MLP. For each bounded scalar coordinate $u\in(0,1)$, compute $z=\lambda\varphi(u)$ and concatenate $[\Psi_{0,\lambda}(u),\ldots,\Psi_{K-1,\lambda}(u)]$ to the ordinary MLP input. Clamp $u$ to $[\epsilon,1-\epsilon]$ and clamp $z$ to a safe range before evaluating the basis. Compute Hermite values using the stable recurrence `psi[0]=pi**(-.25)*exp(-.5*z*z)`, `psi[1]=sqrt(2)*z*psi[0]`, and `psi[k+1]=sqrt(2/(k+1))*z*psi[k]-sqrt(k/(k+1))*psi[k-1]`. Then set `features=sqrt(lambda)*psi` and evaluate `y=MLP(concat(raw_input,features))`. Initialize `log_lambda` from a logarithmic sweep and optionally learn it jointly under bounds. For derivative supervision, use the paper's sparse coefficient rule: compute `b[k]=sqrt((k+1)/2)*a[k+1]-sqrt(k/2)*a[k-1]`, multiply the resulting basis sum by `lambda*phi_prime(u)`, and avoid differentiating each feature through a long automatic-differentiation graph. The mathematical pieces are the orthonormal basis, monotone transform, scale, and sparse derivative recurrence; transform clipping, scale bounds, and hidden-layer width are empirical choices. First test a 1D coordinate MLP on 1,000 samples of $f(u)=\sqrt{u}+\log(1+u)+\sqrt{1-u}$, comparing raw coordinates, Fourier features, Legendre features, and transformed Hermite features at equal parameter count. Sweep $K\in\{8,16,32\}$ and fixed versus learned $\lambda$. Success means lower endpoint-inclusive test error and a steeper error-versus-$K$ curve; a concrete target is 2x lower error than Fourier features at $K=16$ or matching baseline error with half the hidden parameters.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.