# Legendre-polynomial feature trunk

- ID: 695
- Canonical URL: https://synthcore.org/idea/695/legendre-polynomial-feature-trunk
- API JSON: https://synthcore.org/api/idea/695.json
- API Markdown: https://synthcore.org/api/idea/695.md
- Verification status: unverified
- Source: [arXiv:2607.10589](https://arxiv.org/abs/2607.10589)
- Category: architecture
- Solves: accuracy, speedup, scalability
- ML areas: mlp, embedding, initialization
- Math tags: approximation-theory, polynomial, linear-algebra, numerical-analysis
- Ratings: usefulness 6/10; difficulty 6/10; novelty 6/10

## Idea description

Construct a reusable ReLU trunk that emits approximate univariate powers or Legendre-polynomial features for each input coordinate, then combine them with a linear or low-rank polynomial head. This gives a compact explicit basis for smooth functions and can replace a large generic MLP in low-dimensional scientific regression or serve as a frozen or partially trainable front-end.

## Mathematical statement

Lemma 11 constructs a ReLU network f_{NN,1 to nu}^{(Lgd)} from R to R^nu whose i-th output approximates the degree-i Legendre polynomial L_i(x) on [-1,1]. Its width is 2^{ceil(log_2 nu)} times 10N plus 2, its depth is 5 times ceil(log_2 nu)^2 times L, and its coordinatewise error is at most (15/2) times 16^nu times N^{-5L}. The proof uses the intermediate power-feature guarantee |psi_j^{(i)}(x)-x^j| <= 30 times 4^{i-1} times N^{-5L} for j in [2^i]. For a d-dimensional input, use tensor-product basis functions prod_{r=1}^d L_{nu_r}(x_r), with L_0(x)=1, and learn coefficients over a truncated multi-index set Lambda. Here nu is the maximum univariate degree, N controls the approximation width, L controls repeated approximation depth, d is input dimension, and psi_j^{(i)} is the approximate power feature produced at construction stage i.

## Key formulas

- $$\left|f_{NN,i}^{(Lgd)}(x)-L_i(x)\right|\leq\frac{15}{2}\cdot16^{\nu}N^{-5L}$$
- $$\operatorname{width}=2^{\left\lceil\log_{2}\nu\right\rceil}\cdot 10N+2,\qquad \operatorname{depth}=5\left\lceil\log_{2}\nu\right\rceil^{2}L$$
- $$\left|\psi_j^{(i)}(x)-x^j\right|\leq30\cdot4^{i-1}N^{-5L},\qquad j\in[2^i]$$
- $$f(x_1,\ldots,x_d)\approx\sum_{\boldsymbol{\nu}\in\Lambda}a_{\boldsymbol{\nu}}\prod_{r=1}^{d}L_{\nu_r}(x_r),\qquad L_0(x)=1$$

## Implementation notes

Insert the module immediately after input normalization and before the learned prediction head. For each coordinate x_r, pass it through a shared or separately parameterized ReLU polynomial trunk producing phi_r = [1, L_1(x_r), ..., L_nu(x_r)]. For an MVP, train this trunk against analytically computed Legendre targets rather than reproducing every constant in the paper's construction. Select a truncated multi-index set Lambda, for example all nu-vectors satisfying ||nu||_1 <= K, to avoid exponential tensor-product growth. The forward pass is: normalize x to [-1,1]; for each coordinate r compute phi_r = ReLUTrunk_r(x[:,r]); for each multi-index nu in Lambda compute z_nu = product over r of phi_r[:, nu_r]; predict y = Linear(z). During pretraining, minimize sum over r and j of squared error between phi_{r,j} and L_j(x_r). During task training, either freeze the trunk, fine-tune it with a learning rate 10 times smaller than the head, or add the basis loss with coefficient alpha. The paper predicts feature error of order 16^nu N^{-5L}; measure this directly on a dense validation grid and check whether increasing L gives exponential error reduction. The first experiment should use d=2 or 3, nu=8 or 16, and targets exp(x_1+x_2), sin(x_1 x_2), and a smooth elliptic-PDE solution. Compare against a parameter-matched ReLU MLP and a fixed analytic Legendre-feature baseline. Success is lower test and gridwise error at equal parameters, or the same error with fewer hidden parameters and faster convergence.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.