# Residual-Christoffel Collocation for Random-Feature PDE Networks

- ID: 858
- Canonical URL: https://synthcore.org/idea/858/residual-christoffel-collocation-for-random-feature-pde-netw
- API JSON: https://synthcore.org/api/idea/858.json
- API Markdown: https://synthcore.org/api/idea/858.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.13382](https://arxiv.org/abs/2607.13382)
- Category: training
- Solves: stability, speedup, accuracy
- ML areas: mlp, training-dynamics, loss, initialization
- Math tags: linear-algebra, spectral-theory, approximation-theory, numerical-analysis
- Ratings: usefulness 8/10; difficulty 5/10; novelty 7/10

## Idea description

Replace uniform collocation for a fixed random-feature neural PDE solver with sampling from the leverage-score density of the operator-applied features. Whiten the retained residual feature space before solving for output coefficients, so the sampled least-squares matrix has an identity-like expected Gram rather than inheriting severe anisotropy from the differential operator. The same construction can be used for a linearized neural network by treating Jacobian features as the trial functions.

## Mathematical statement

For fixed trial functions $\{\varphi_m\}_{m=1}^M$, the paper defines the residual feature vector $\psi(x)=(L\varphi_1(x),\ldots,L\varphi_M(x))^\top$, where $L$ is the interior PDE operator. Let $\mu$ be a reference sampling measure on the interior, or a mixture of interior and boundary measures when boundary rows are included. The reference residual Gram is $G=\int_\Omega\psi(x)\psi(x)^\top d\mu(x)$. With eigendecomposition $G=U\Lambda U^\top$, retain the $r$ eigenvectors whose eigenvalues exceed threshold $\tau$; denote the retained factors by $U_r$ and $\Lambda_r$. Define whitened residual features $z(x)=\Lambda_r^{-1/2}U_r^\top\psi(x)$ and the residual-Christoffel density $q(x)=\|z(x)\|_2^2/r$ relative to $\mu$. Since $\int q\,d\mu=1$, sampling $x_i\sim q(x)d\mu(x)$ and multiplying each row by $(Nq(x_i))^{-1/2}$ gives $\mathbb{E}[A^\top A]=I_r$ for $A_i=(Nq(x_i))^{-1/2}z(x_i)^\top$. The transformed coefficients satisfy $c=U_r\Lambda_r^{-1/2}\theta$, so the PDE residual is $\psi(x)^\top c=z(x)^\top\theta$. A ridge-stable alternative uses $G_\lambda=G+\lambda I$, effective dimension $d_\lambda=\operatorname{tr}(GG_\lambda^{-1})$, and $q_\lambda(x)=\|G_\lambda^{-1/2}\psi(x)\|_2^2/d_\lambda$.

## Key formulas

- $$u_M(x)=\sum_{m=1}^{M}c_m\varphi_m(x),\qquad \psi(x)=\big(L\varphi_1(x),\ldots,L\varphi_M(x)\big)^\top.$$
- $$G=\int_\Omega\psi(x)\psi(x)^\top\,d\mu(x)=U\Lambda U^\top,\qquad z(x)=\Lambda_r^{-1/2}U_r^\top\psi(x),\qquad q(x)=\frac{\|z(x)\|_2^2}{r}.$$
- $$x_i\sim q(x)\,d\mu(x),\qquad A_i=\frac{z(x_i)^\top}{\sqrt{Nq(x_i)}},\qquad b_i=\frac{f(x_i)}{\sqrt{Nq(x_i)}},\qquad \mathbb{E}[A^\top A]=I_r.$$
- $$G_\lambda=G+\lambda I,\qquad z_\lambda(x)=G_\lambda^{-1/2}\psi(x),\qquad d_\lambda=\operatorname{tr}\!\left(GG_\lambda^{-1}\right),\qquad q_\lambda(x)=\frac{\|z_\lambda(x)\|_2^2}{d_\lambda}.$$

## Implementation notes

1. Integration point: use a fixed random-feature MLP or ELM for a PDE, with trainable output coefficients only. For each candidate point, compute the feature vector $\varphi(x)$ and its differentiated vector $\psi(x)=L\varphi(x)$ using automatic differentiation. If enforcing boundary conditions strongly is impossible, concatenate interior rows $L\varphi(x)$ and boundary rows $B\varphi(x)$, and define $\mu$ as a mixture with an explicit interior/boundary sampling probability.

2. Pseudocode: draw a pilot set $\{\tilde x_j\}_{j=1}^P$ uniformly from $\mu$; form $\Psi_j=\psi(\tilde x_j)$ and estimate $G\leftarrow P^{-1}\sum_j\Psi_j\Psi_j^\top+\epsilon I$. Either eigendecompose $G$ and retain $\lambda_k\ge\tau$, then compute $z_j=\Lambda_r^{-1/2}U_r^\top\Psi_j$ and scores $s_j=\|z_j\|^2/r$, or use the ridge version $z_j=(G+\lambda I)^{-1/2}\Psi_j$ and $s_j=\|z_j\|^2/d_\lambda$. Normalize scores over the pilot set, sample collocation indices proportional to them, evaluate fresh rows, multiply row $i$ and target by $1/\sqrt{Nq_i}$, and solve the whitened least-squares problem for $\theta$. Recover output coefficients with $c=U_r\Lambda_r^{-1/2}\theta$.

3. Computed from the mathematics: the Gram estimate, eigenspace or ridge inverse square root, Christoffel scores, inverse-density weights, and whitened coefficient parameterization. Estimated empirically: the pilot approximation to $G$, the threshold $\tau$ or ridge $\lambda$, and the mixture weight between interior and boundary rows. Clip $q_i$ below by $q_{\min}$ to avoid exploding weights, and compare clipping levels in an ablation.

4. First experiment: solve 1D Poisson, 2D Poisson, and a variable-coefficient elliptic equation using the same 2-layer random-feature MLP and identical feature initialization. Compare uniform collocation, residual-magnitude adaptive sampling, residual-Christoffel sampling without whitening, and the full weighted-whitened method. Sweep the number of collocation points and record smallest singular value, condition number, QR or conjugate-gradient iterations, relative $L^2$ solution error, and PDE residual. The expected signal is a flatter condition-number curve and lower error at small sample counts; a compelling result would be reaching a fixed relative error with 2x fewer rows or reducing linear-solve iterations by at least 2x at equal feature count.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a fixed random-feature 1D Poisson solver with residual-Christoffel sampling, inverse-density weighting, eigenspace whitening, and a uniform-collocation baseline. The mechanism was confirmed: q integrated to 1.000, the population whitened Gram error was 3.97e-9, the direct q-sampling expectation error was 0.0286, and finite-sample Gram error followed the predicted N^-1/2 law with fitted slope -0.496. Conditioning improved dramatically, but PDE L2 solution error was worse than uniform sampling in this tiny setup, likely because boundary rows and aggressive residual weighting were not tuned.

### Mechanism check

- Verdict: Built a fixed random-feature 1D Poisson solver with residual-Christoffel sampling, inverse-density weighting, eigenspace whitening, and a uniform-collocation baseline. The mechanism was confirmed: q integrated to 1.000, the population whitened Gram error was 3.97e-9, the direct q-sampling expectation error was 0.0286, and finite-sample Gram error followed the predicted N^-1/2 law with fitted slope -0.496. Conditioning improved dramatically, but PDE L2 solution error was worse than uniform sampling in this tiny setup, likely because boundary rows and aggressive residual weighting were not tuned.
- Confidence: 8/10
- Limitations: Only a 1D Poisson problem and analytic tanh second derivatives were tested; no 2D or variable-coefficient PDE, autodiff Jacobian features, residual-magnitude adaptive baseline, q clipping ablation, timing/FLOP comparison, or independently estimated pilot Gram was evaluated. Boundary weighting was fixed rather than tuned, and the pilot distribution used a dense quadrature grid.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/159/bench_report.json)
- [bench_run.py](https://synthcore.org/code/159/bench_run.py)
- [experiment.py](https://synthcore.org/code/159/experiment.py)
- [poisson_dirichlet_local.py](https://synthcore.org/code/159/poisson_dirichlet_local.py)
- [report.md](https://synthcore.org/code/159/report.md)
- [report_bench_2026-08-30T182101.md](https://synthcore.org/code/159/report_bench_2026-08-30T182101.md)
- [results.json](https://synthcore.org/code/159/results.json)
- [Download all files as ZIP](https://synthcore.org/download/159)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.