# Inverse-Laplacian Residual Loss

- ID: 373
- Canonical URL: https://synthcore.org/idea/373/inverse-laplacian-residual-loss
- API JSON: https://synthcore.org/api/idea/373.json
- API Markdown: https://synthcore.org/api/idea/373.md
- Verification status: mechanism_failed
- Source: [arXiv:2607.02726](https://arxiv.org/abs/2607.02726)
- Category: training
- Solves: stability, speedup, accuracy
- ML areas: loss, training-dynamics, optimizer
- Math tags: functional-analysis, pde, numerical-analysis, linear-algebra
- Ratings: usefulness 8/10; difficulty 5/10; novelty 7/10

## Idea description

Replace the standard squared pointwise PDE residual in an elliptic PINN by its discrete $H^{-1}$ norm. The residual is passed through an inverse Dirichlet Laplacian, reducing the dominance of rapidly varying residual modes and acting as a mathematically specified preconditioner for the PINN training gradients.

## Mathematical statement

The paper starts from the standard residual objective $\mathcal{L}(\theta)=\mathcal{L}_{\Omega}(\theta)+\mathcal{L}_{\partial\Omega}(\theta)$ with $\mathcal{L}_{\Omega}(\theta)=\int_{\Omega}(\mathcal{N}(u_{\theta})-f)^2\,d^m\!x$ and a boundary residual term. For an elliptic residual $r_{\theta}(x)=\mathcal{N}(u_{\theta})(x)-f(x)$, the adapted $H^{-1}$ loss is $\|r_{\theta}\|_{H^{-1}(\Omega)}^2=\langle r_{\theta},(-\Delta_D)^{-1}r_{\theta}\rangle$, where $-\Delta_D$ is the positive Laplacian with homogeneous Dirichlet boundary conditions. Equivalently, define $\phi_{\theta}$ by $-\Delta\phi_{\theta}=r_{\theta}$ in $\Omega$ and $\phi_{\theta}=0$ on $\partial\Omega$; then $\|r_{\theta}\|_{H^{-1}}^2=\int_{\Omega}|\nabla\phi_{\theta}|^2dx$. In a finite-element or finite-difference discretization with mass matrix $M$ and stiffness matrix $K$, the implementable loss is $r^\top M K^{-1}M r$. The paper combines an $H^{-1}$ interior residual with an $H^{1/2}$ boundary residual using a Lions--Magenes estimate to obtain a formulation continuous in the $H^1$ norm; if exact fractional boundary norms are unavailable, strongly imposing Dirichlet data through a lifting is a simpler controlled approximation.

## Key formulas

- $$\mathcal{L}(\bm{\theta})\coloneqq\mathcal{L}_{\Omega}(\bm{\theta})+\mathcal{L}_{\partial\Omega}(\bm{\theta})\coloneqq\int_{\Omega}(\mathcal{N}(u_{\bm{\theta}})-f)^2d^m\bm{x}+\int_{\partial\Omega}(u_{\bm{\theta}}|_{\partial\Omega}-g)^2\,dS.$$
- $$\|r_{\theta}\|_{H^{-1}(\Omega)}^2=\left\langle r_{\theta},(-\Delta_D)^{-1}r_{\theta}\right\rangle=\int_{\Omega}|\nabla\phi_{\theta}(x)|^2\,dx,\qquad -\Delta\phi_{\theta}=r_{\theta},\quad \phi_{\theta}|_{\partial\Omega}=0.$$
- $$\mathcal{L}_{H^{-1}}(\theta)=r_\theta^{\top}M K^{-1}M r_\theta,\qquad K z=M r_\theta,\qquad \mathcal{L}_{H^{-1}}(\theta)=r_\theta^{\top}M z.$$
- $$u_\theta(x)=g_{\mathrm{ext}}(x)+d(x)v_\theta(x),\qquad d|_{\partial\Omega}=0.$$

## Implementation notes

(1) Integration point: modify the PDE-loss branch of a PINN training step, not the network architecture. For a batch of fixed interior grid points, evaluate $u_\theta$, use automatic differentiation to compute the PDE residual vector $r_\theta=[\mathcal{N}(u_\theta)(x_i)-f(x_i)]_{i=1}^n$, and replace its MSE by a discrete inverse-Laplacian quadratic form. Dirichlet data should initially be imposed exactly with $u_\theta(x)=g_{\mathrm{ext}}(x)+d(x)v_\theta(x)$, where $d$ vanishes on the boundary; this avoids mixing an unpreconditioned boundary MSE with the filtered interior loss. (2) Pseudocode: precompute a finite-difference or finite-element mass matrix $M$ and Dirichlet stiffness matrix $K$ on the collocation grid; at each step compute `r = pde_residual(net, x_grid)`, solve `z = solve(K, M @ r)`, set `loss_pde = r.T @ M @ z`, add any data or initial-condition losses, backpropagate, and update Adam or LBFGS. Use a Cholesky factorization of $K$ for small fixed grids, or conjugate gradients with a fixed tolerance for larger grids. Detach neither `z` nor the solve output if gradients through the loss are desired; alternatively use the symmetric quadratic-form gradient $2J_r^\top M K^{-1}M r$. (3) The paper supplies the $H^{-1}$ construction and its elliptic smoothing rationale. The engineer estimates $M$ and $K$ numerically from the domain and chooses the grid resolution; verify the solve by checking $\|Kz-Mr\|/\|Mr\|$. Normalize by the number of grid points and optionally add a small ridge $K+\varepsilon M$ only if the linear solve is poorly conditioned. (4) First experiment: solve 2D Poisson $-\Delta u=f$ on $[0,1]^2$ with nonzero Dirichlet data using the same MLP, collocation budget, optimizer, and random seeds for standard residual MSE versus $H^{-1}$ loss. Compare relative $L^2$ and $H^1$ solution errors, gradient-norm variance, residual-spectrum decay, wall-clock time, and iterations to a fixed error. Success means faster loss descent or fewer optimizer steps at equal PDE evaluations, reduced sensitivity to the interior/boundary scale ratio, and lower $H^1$ error; a useful ablation compares exact $K^{-1}$ against one or three approximate CG iterations.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a matched 2D Poisson PINN benchmark with exact Dirichlet lifting, standard residual MSE, and a differentiable discrete inverse-Laplacian quadratic form. The math check passed: solve relative residual was 1.07e-15 and the lowest-mode eigenvalue scaling error was 1.64e-15. In this fixed-budget run, H^-1 reduced gradient variability but was slower and substantially worse in solution accuracy than MSE, so the promised training or accuracy win was not observed.

### Mechanism check

- Verdict: Built a matched 2D Poisson PINN benchmark with exact Dirichlet lifting, standard residual MSE, and a differentiable discrete inverse-Laplacian quadratic form. The math check passed: solve relative residual was 1.07e-15 and the lowest-mode eigenvalue scaling error was 1.64e-15. In this fixed-budget run, H^-1 reduced gradient variability but was slower and substantially worse in solution accuracy than MSE, so the promised training or accuracy win was not observed.
- Confidence: 9/10
- Limitations: Only one manufactured 2D Poisson problem, one grid resolution, one optimizer learning rate, one seed, and exact dense solves were tested. No approximate CG ablation, boundary-scale sensitivity study, larger-grid study, or repeated-seed statistical comparison was performed.

## Artifacts

- [report.md](https://synthcore.org/code/56/report.md)
- [results.json](https://synthcore.org/code/56/results.json)
- [run_experiment.py](https://synthcore.org/code/56/run_experiment.py)
- [Download all files as ZIP](https://synthcore.org/download/56)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.