# Derivative-Free Very-Weak Neural PDE Solver

- ID: 916
- Canonical URL: https://synthcore.org/idea/916/derivative-free-very-weak-neural-pde-solver
- API JSON: https://synthcore.org/api/idea/916.json
- API Markdown: https://synthcore.org/api/idea/916.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.14498](https://arxiv.org/abs/2607.14498)
- Category: training
- Solves: speedup, stability, scalability
- ML areas: mlp, training, inference-speedup
- Math tags: pde, functional-analysis, numerical-analysis
- Ratings: usefulness 8/10; difficulty 4/10; novelty 6/10

## Idea description

Train a neural trial function for an elliptic PDE using a very-weak residual in which all derivatives act on fixed smooth test functions rather than on the neural network. This eliminates second-order reverse-mode or forward-mode automatic differentiation and allows low-regularity activations while retaining a least-squares objective over many test functions.

## Mathematical statement

For the Dirichlet problem $-\Delta u=f$ in $\Omega$ with $u=g$ on $\Gamma=\partial\Omega$, the paper gives, for every test function $w\in H^2(\Omega)\cap H_0^1(\Omega)$, the identity $-\int_\Omega u\Delta w=-\int_\Gamma g\partial_\eta w+\int_\Omega fw$. Here $u$ is the PDE solution, $f$ is the forcing term, $g$ is the boundary value, $\eta$ is the outward unit normal, and $\partial_\eta w=\nabla w\cdot\eta$. Given smooth tests $w_j$, define $u_\theta$ as the neural approximation and $r_j(\theta)=-\int_\Omega u_\theta\Delta w_j+\int_\Gamma g\partial_\eta w_j-\int_\Omega fw_j$. The adapted least-squares loss is $L(\theta)=m^{-1}\sum_{j=1}^m r_j(\theta)^2$ plus an optional boundary penalty. Since only $u_\theta$ is evaluated, no spatial derivatives of the neural network are required.

## Key formulas

- $$-\Delta u=f\ \text{in }\Omega,\qquad u=g\ \text{on }\Gamma=\partial\Omega.$$
- $$-\int_{\Omega}u\Delta w=-\int_{\Gamma}g\frac{\partial w}{\partial\eta}+\int_{\Omega}fw,\qquad w\in H^2(\Omega)\cap H_0^1(\Omega).$$
- $$r_j(\theta)=-\int_{\Omega}u_\theta(x)\Delta w_j(x)\,dx+\int_{\Gamma}g(s)\partial_\eta w_j(s)\,ds-\int_{\Omega}f(x)w_j(x)\,dx.$$
- $$L(\theta)=\frac{1}{m}\sum_{j=1}^{m}r_j(\theta)^2+\lambda\,\mathbb{E}_{x\sim\Gamma}\left|u_\theta(x)-g(x)\right|^2.$$

## Implementation notes

Integrate this at the PDE loss layer, replacing the usual pointwise residual $|\Delta u_\theta-f|^2$, which requires differentiating the network twice. Use a network $v_\theta(x)$ and, when possible, a boundary-preserving parameterization $u_\theta(x)=\ell(x)+d(x)v_\theta(x)$, where $\ell$ extends $g$ into the domain and $d$ vanishes on the boundary. Choose $m$ fixed test functions $w_j$ that vanish on the boundary and are twice differentiable; sine products on a box, compactly supported smooth bumps, or low-order finite-element basis functions are suitable. Precompute quadrature points $x_q$, weights $a_q$, values $w_j(x_q)$, $\Delta w_j(x_q)$, and boundary values $\partial_\eta w_j(s_q)$. At each iteration, evaluate only $u_\theta(x_q)$ and compute `r[j] = -sum_q(a[q] * u[q] * lap_w[j,q]) + boundary_term[j] - forcing_term[j]`; backpropagate through the scalar network outputs but never call input-Jacobian or Hessian autodiff. Randomly rotate or resample the test-function subset every few epochs to reduce the risk of fitting a fixed projection nullspace. The PDE identity and quadrature expressions are mathematical components; quadrature resolution, test selection, and the boundary penalty are empirical choices. First test Poisson on a 2D unit square with a known sinusoidal solution using a four-layer MLP. Compare against a standard PINN with second-order autodiff at equal wall-clock time and equal width. Measure loss descent per second, peak memory, final relative $L^2$ error, and boundary error. Success means comparable or lower error with at least 2x lower training memory or time, especially for high-dimensional inputs or nonsmooth activations.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a derivative-free very-weak Poisson solver with precomputed sine-test Laplacians and a boundary-preserving ReLU MLP, plus a genuine second-order autodiff strong-form baseline. Exact residuals stayed below 4e-15 across quadrature orders and residual/(delta·eigenvalue) was 1.0 across four modes. Weak ReLU achieved 0.0157 relative L2 error versus 0.278 for strong ReLU and ran 6.8x faster, although memory savings were not visible for this tiny model.

### Mechanism check

- Verdict: Built a derivative-free very-weak Poisson solver with precomputed sine-test Laplacians and a boundary-preserving ReLU MLP, plus a genuine second-order autodiff strong-form baseline. Exact residuals stayed below 4e-15 across quadrature orders and residual/(delta·eigenvalue) was 1.0 across four modes. Weak ReLU achieved 0.0157 relative L2 error versus 0.278 for strong ReLU and ran 6.8x faster, although memory savings were not visible for this tiny model.
- Confidence: 8/10
- Limitations: The comparison used a small 2D manufactured Poisson problem, fixed sine tests, boundary-preserving parameterization, equal optimizer steps rather than equal FLOPs, and a tiny network where memory differences are not meaningful. It did not test higher dimensions, irregular domains, nonzero boundary data, adaptive or random test resampling, nonsmooth forcing, or large-scale memory behavior.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/166/bench_report.json)
- [bench_stage2.py](https://synthcore.org/code/166/bench_stage2.py)
- [experiment.py](https://synthcore.org/code/166/experiment.py)
- [report.md](https://synthcore.org/code/166/report.md)
- [report_bench_2026-08-30T182259.md](https://synthcore.org/code/166/report_bench_2026-08-30T182259.md)
- [results.json](https://synthcore.org/code/166/results.json)
- [Download all files as ZIP](https://synthcore.org/download/166)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.