# Occupation-weighted Hessian contraction for PINNs

- ID: 2384
- Canonical URL: https://synthcore.org/idea/2384/occupation-weighted-hessian-contraction-for-pinns
- API JSON: https://synthcore.org/api/idea/2384.json
- API Markdown: https://synthcore.org/api/idea/2384.md
- Verification status: unverified
- Source: [arXiv:2608.16119](https://arxiv.org/abs/2608.16119)
- Category: training
- Solves: stability, sample-efficiency, accuracy
- ML areas: training, loss, world-model
- Math tags: pde, stochastic-processes, geometry, functional-analysis, dynamical-systems
- Ratings: usefulness 6/10; difficulty 5/10; novelty 8/10

## Idea description

Train a neural PDE solver using collocation points sampled from a fixed reference diffusion and a time weight that compensates for the point-start singularity. Replace the Euclidean Hessian by the intrinsic tensor Gθ=σD²uθσ, and use source Picard updates so that nonlinear curvature coupling is iterated under an explicit contraction target.

## Mathematical statement

The reference diffusion is dX_t=b(t,X_t)dt+σ(t,X_t)dW_t, with covariance a=σσ⊤ and generator L_t=b·D_x+(1/2)a:D_x². The paper defines the intrinsic derivatives Z u=σD_xu and G u=σD_x²uσ, and the backward operator P=-(∂_t+L_t). Let μ_t be the law of X_t and let ρ_w(dt,dx)=w(t)dtμ_t(dx), where w(t)>0 is a time weight. For the zero-terminal linear problem Pv=q, v(T,·)=0, C_w(X) is the optimal source-to-intrinsic-Hessian norm satisfying ||Gv||_{ρ_w}≤C_w(X)||q||_{ρ_w}. If the nonlinear driver f(t,x,u,Zu,Gu) is L_H-Lipschitz in its Hessian argument, source Picard iteration contracts when C_w(X)L_H<1. In the Brownian case the supplied benchmark is C_α^Br=sup_{n≥2}2√n(n-1)/(n-1+α), for a weight of order α.

## Key formulas

- $$\mathrm{d}X_t=b(t,X_t)\,\mathrm{d}t+\sigma(t,X_t)\,\mathrm{d}W_t,\qquad a=\sigma\sigma^{\top},\qquad \mathcal{L}_t=b\cdot D_x+\frac{1}{2}a:D_x^2,$$
- $$\mathcal{P}u=-(\partial_t+\mathcal{L}_t)u,\qquad \mathsf{Z}u=\sigma D_xu,\qquad \mathsf{G}u=\sigma D_x^2u\,\sigma,$$
- $$\rho_w(\mathrm{d}t,\mathrm{d}x)=w(t)\,\mathrm{d}t\,\mu_t(\mathrm{d}x),\qquad \|\mathsf{G}v\|_{\rho_w}\le C_w(X)\|q\|_{\rho_w}\quad\text{when }\mathcal{P}v=q,\ v(T,\cdot)=0,$$
- $$C_w(X)L_H<1,\qquad C_{\alpha}^{\mathrm{Br}}=\sup_{n\in\mathbb{N},\,n\ge 2}\frac{2\sqrt{n(n-1)}}{n-1+\alpha}.$$

## Implementation notes

(1) Integration point: modify the residual sampler and curvature residual of a terminal-value PINN. Let uθ(t,x) be an MLP and let fφ(t,x,u,z,g) be the PDE driver. Compute D_tuθ, D_xuθ, and D_x²uθ by automatic differentiation, then form zθ=σD_xuθ and gθ=σD_x²uθσ. Sample x at time t from trajectories of the fixed reference SDE instead of from a uniform spatial grid. Use w(t)=t^{α-1}, initially α=1/2, with importance correction if samples are drawn from another time proposal.

(2) Pseudocode: simulate X[j] with Euler-Maruyama from x₀; sample time index j with probability proportional to w(t[j]); evaluate r=fφ(t[j],X[j],uθ,zθ,gθ)+D_tuθ+b·D_xuθ+(1/2)a:D_x²uθ; minimize the weighted loss w(t[j])|r|² plus terminal loss |uθ(T,x)-ψ(x)|². For source Picard, initialize q⁰=0, fit a network vᵏ to Pvᵏ=qᵏ, compute qᵏ⁺¹=fφ(t,x,vᵏ,Zvᵏ,Gvᵏ) with vᵏ detached while fitting the next source, and stop when the weighted source difference is below tolerance.

(3) Compute the intrinsic derivatives and occupation norm directly. Compute C_α^Br from the displayed supremum. Estimate L_H by drawing matched tuples (t,x,u,z) and two Hessians g₁,g₂, using L̂_H=max ||fφ(...,g₁)-fφ(...,g₂)||/(||g₁-g₂||+ε). Track the margin m=1-C_α^BrL̂_H. If m≤0, reduce the g-channel scale or apply spectral normalization to that channel; do not interpret this as a proof for arbitrary learned drivers. Compare against uniform collocation, Euclidean Hessians, and direct residual training without Picard iteration.

(4) First experiment: solve a two-dimensional Allen-Cahn or HJB terminal-value PDE with a small tanh MLP. Use identical width, parameter count, collocation budget, and optimizer for uniform PINN and occupation-weighted PINN, with and without Picard source updates. Evaluate dense-grid relative solution error, weighted PDE residual, terminal error, optimizer steps to a fixed error, and the empirical Picard ratio ||qᵏ⁺¹-qᵏ||/|qᵏ-qᵏ⁻¹|. Success means improved behavior near t=0, a ratio below one when the estimated threshold holds, and at least 1.5x fewer steps or lower error at equal collocation cost.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.