# Localized Petrov–Galerkin Neural Residuals

- ID: 1188
- Canonical URL: https://synthcore.org/idea/1188/localized-petrov-galerkin-neural-residuals
- API JSON: https://synthcore.org/api/idea/1188.json
- API Markdown: https://synthcore.org/api/idea/1188.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.20378](https://arxiv.org/abs/2607.20378)
- Category: training
- Solves: stability, accuracy, sample-efficiency
- ML areas: loss, mlp, optimizer, training-dynamics
- Math tags: pde, numerical-analysis, functional-analysis
- Ratings: usefulness 8/10; difficulty 5/10; novelty 5/10

## Idea description

Replace the pointwise strong-form PINN loss with a vector of localized weak residuals generated by fixed compactly supported polynomial test functions. Use a neural network or KAN as the trial function, integrate by parts once, and evaluate each test residual with Gauss–Legendre quadrature; this lowers the required derivative order and prevents a few high-curvature collocation points from dominating training.

## Mathematical statement

The paper considers the second-order boundary-value operator \(\mathcal{P}(u)=-\nabla\!\cdot(\bm{\kappa}\nabla u)+\bm{b}\cdot\nabla u+c u=f\) in \(\Omega\), with Dirichlet data \(u=\bar u\) on \(\Gamma^u\) and flux data \((\bm{\kappa}\nabla u)\cdot\bm n=\bar t\) on \(\Gamma^t\). Here \(u_\theta\) is the neural trial function, \(\bm\kappa(x)\) is a possibly anisotropic spatial coefficient, \(\bm b(x)\) is advection, \(c(x)\) is reaction, \(f(x)\) is forcing, and \(\bm n\) is the outward normal. For a test function \(v_e\) that vanishes on \(\Gamma^u\), integration by parts gives the localized weak residual \(R_e(\theta)=\int_e[\nabla u_\theta^\top\bm\kappa\nabla v_e+v_e\bm b\cdot\nabla u_\theta+c u_\theta v_e-fv_e]dx-\int_{\Gamma^t\cap\partial e}\bar t v_e ds\). Choosing \(v_e\) compactly supported in element \(e\) localizes the residual, and Gauss–Legendre quadrature approximates it using only first derivatives of \(u_\theta\). The loss is \(\mathcal L_{PG}=\sum_e\omega_eR_e^2+\lambda_D\mathcal L_D+\lambda_N\mathcal L_N\), where \(\omega_e\) are element weights and the boundary terms enforce Dirichlet and Neumann conditions.

## Key formulas

- $$\mathcal{P}(u)=-\nabla\!\cdot\big(\bm{\kappa}\,\nabla u\big)+\bm{b}\cdot\nabla u+c\,u=f\ \text{in }\Omega,\qquad u=\bar{u}\ \text{on }\Gamma^{u},\qquad(\bm{\kappa}\nabla u)\cdot\bm{n}=\bar{t}\ \text{on }\Gamma^{t}.$$
- $$R_e(\theta)=\int_{e}\left[\nabla u_\theta^{\top}\bm{\kappa}\nabla v_e+v_e\,\bm{b}\cdot\nabla u_\theta+c\,u_\theta v_e-fv_e\right]dx-\int_{\Gamma^t\cap\partial e}\bar t\,v_e\,ds.$$
- $$R_e(\theta)\approx |J_e|\sum_{q=1}^{Q}w_q\left[\nabla u_\theta(x_{eq})^{\top}\bm\kappa(x_{eq})\nabla v_e(x_{eq})+v_e(x_{eq})\bm b(x_{eq})\cdot\nabla u_\theta(x_{eq})+c(x_{eq})u_\theta(x_{eq})v_e(x_{eq})-f(x_{eq})v_e(x_{eq})\right].$$
- $$\mathcal L_{PG}=\sum_{e=1}^{E}\omega_eR_e(\theta)^2+\lambda_D\frac{1}{|\Gamma^u|}\sum_{x_i\in\Gamma^u}\|u_\theta(x_i)-\bar u(x_i)\|^2+\lambda_N\frac{1}{|\Gamma^t|}\sum_{x_i\in\Gamma^t}\|\bm\kappa\nabla u_\theta(x_i)\cdot\bm n-\bar t(x_i)\|^2.$$

## Implementation notes

1. Integration point: modify the physics loss of a PINN or KAN, not the network forward pass. Partition the domain into 1D cells, 2D quadrilaterals or triangles, or small axis-aligned patches. For every element \(e\), define one or several fixed compactly supported test functions \(v_e\), such as linear hat functions or local Legendre polynomials that vanish outside the element. Precompute \(v_e\), \(\nabla v_e\), Gauss points \(x_{eq}\), weights \(w_q\), and Jacobian determinants \(|J_e|\). 2. Pseudocode: `theta=init(); for step in range(T): u=net(x_g); grad_u=autograd(u,x_g); term=dot(grad_u,kappa(x_g)*grad_v)+v*(b(x_g)*grad_u)+c(x_g)*u*v-f(x_g)*v; R[e]=sum_q(w[q]*detJ[e]*term[e,q]); loss=sum_e omega[e]*R[e]**2 + lambdaD*mse(net(xD),uD) + lambdaN*mse(flux(xN),tN); theta=optimizer_step(loss,theta)`. Use first derivatives only and do not compute the strong residual \(-\nabla\cdot(\kappa\nabla u)\). 3. The weak residual and quadrature rule come from the mathematical construction; \(\kappa,\bm b,c,f\) are supplied by the PDE or represented by differentiable coefficient networks. Estimate optional weights empirically with \(\omega_e=1/(\operatorname{EMA}(R_e^2)+\epsilon)\), clipped to \([0.1,10]\), or use uniform weights initially. For inverse problems, replace \(\kappa\) by \(\kappa_\phi\) and evaluate residuals using the measured field \(u_{obs}\), while omitting boundary penalties from the coefficient-network loss. 4. First experiment: solve a 2D manufactured Poisson or advection–diffusion problem on the unit square, comparing an equal-width MLP strong-form PINN, the same MLP with pointwise residual loss, and this localized weak loss at equal network evaluations. Start with 16 elements, four Gauss points per element, and one linear test per element. Measure relative \(L^2\) error, gradient error, wall-clock time to a fixed error, and residual variance across elements. The expected signal is lower loss variance and faster convergence, especially for nonsmooth or high-frequency solutions; target at least 2x lower error at equal optimization steps or substantially fewer autodiff operations at equal accuracy. Then test inverse recovery of a spatially varying diffusion coefficient and check whether the recovered coefficient avoids the zero-coefficient collapse.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built a localized 1D Petrov–Galerkin neural residual module using compact quadratic bubble tests, Gauss–Legendre quadrature, and only first derivatives of the trial network. Integration by parts matched to 9.8e-15 error, quadrature became exact at Q>=5 for the tested degree-8 case, and phase-averaged high-frequency attenuation had observed slope -1.977 versus predicted -2.0. The weak PINN achieved relative L2 error 1.69e-4 versus 1.28e-4 for strong form, so the mechanism manifested but no accuracy or speed win was observed in this smooth toy problem.

### Mechanism check

- Verdict: Built a localized 1D Petrov–Galerkin neural residual module using compact quadratic bubble tests, Gauss–Legendre quadrature, and only first derivatives of the trial network. Integration by parts matched to 9.8e-15 error, quadrature became exact at Q>=5 for the tested degree-8 case, and phase-averaged high-frequency attenuation had observed slope -1.977 versus predicted -2.0. The weak PINN achieved relative L2 error 1.69e-4 versus 1.28e-4 for strong form, so the mechanism manifested but no accuracy or speed win was observed in this smooth toy problem.
- Confidence: 9/10
- Limitations: Only a 1D smooth manufactured Poisson problem was tested. The experiment did not test 2D geometry, anisotropic or advection operators, nonsmooth or singular solutions, adaptive element weights, KAN trial functions, inverse coefficient recovery, equal-FLOP comparisons, or memory scaling.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/222/bench_report.json)
- [localized_pg_experiment.py](https://synthcore.org/code/222/localized_pg_experiment.py)
- [report.md](https://synthcore.org/code/222/report.md)
- [report_bench_2026-08-30T210443.md](https://synthcore.org/code/222/report_bench_2026-08-30T210443.md)
- [results.json](https://synthcore.org/code/222/results.json)
- [stage2_bench.py](https://synthcore.org/code/222/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/222)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.