✗ Failed on benchmark 2026

Second-Order Brownian Jet Residual

Implementation & benchmark of arXiv:2607.16730 — A Deep Second-Order Stochastic Residual Method for Fully Nonlinear Parabolic PDEs

Usefulness8/10
Difficulty5/10
Novelty7/10

Source paper: A Deep Second-Order Stochastic Residual Method for Fully Nonlinear Parabolic PDEs arXiv:2607.16730 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace pointwise high-order PINN residuals with a stochastic one-step residual evaluated on Brownian transitions. A single scalar network produces the value, gradient, and Hessian by automatic differentiation, and the quadratic centered increment supplies a stochastic probe of the Hessian. Add a terminal gradient penalty so the learned full jet is constrained at the terminal boundary, not only the scalar value.

Formulas

$$\partial_{t}u^{\star}+\frac{1}{2}\Delta u^{\star}+f\!\left(t,x,u^{\star},\nabla_{x}u^{\star},D_{x}^{2}u^{\star}\right)=0,\qquad u^{\star}(T,\cdot)=g.$$
$$\Delta W_i=W_{t_{i+1}}-W_{t_i},\qquad Q_i=\Delta W_i\Delta W_i^{\top}-hI_d.$$
$$R_i=Y_{i+1}-Y_i+h f(t_i,X_i,Y_i,Z_i,\Gamma_i)-Z_i^{\top}\Delta W_i-\frac{1}{2}\Gamma_i:Q_i.$$
$$\mathcal{L}_\theta=\frac{1}{N}\sum_{i=0}^{N-1}\mathbb{E}[R_i^2]+\lambda_y\mathbb{E}\left[|U_\theta(T,X_T)-g(X_T)|^2\right]+\lambda_g\mathbb{E}\left[\|\nabla_xU_\theta(T,X_T)-\nabla g(X_T)\|_2^2\right].$$

Mathematical statement

The target equation is the fully nonlinear parabolic PDE $\partial_t u^{\star}+\frac{1}{2}\Delta u^{\star}+f(t,x,u^{\star},\nabla_xu^{\star},D_x^2u^{\star})=0$ with terminal condition $u^{\star}(T,x)=g(x)$. Let $X_i\in\mathbb{R}^d$ be a Brownian state at $t_i=ih$, let $\Delta W_i=W_{t_{i+1}}-W_{t_i}\sim\mathcal{N}(0,hI_d)$, and define the centered quadratic increment $Q_i=\Delta W_i\Delta W_i^{\top}-hI_d$. For a scalar network $U_\theta(t,x)$, define $Y_i=U_\theta(t_i,X_i)$, $Z_i=\nabla_xU_\theta(t_i,X_i)\in\mathbb{R}^d$, and $\Gamma_i=D_x^2U_\theta(t_i,X_i)\in\mathbb{S}^d$. The second-order stochastic Taylor residual is $R_i=Y_{i+1}-Y_i+h f(t_i,X_i,Y_i,Z_i,\Gamma_i)-Z_i^{\top}\Delta W_i-\frac{1}{2}\Gamma_i:Q_i$, where $A:B=\operatorname{tr}(A^{\top}B)$ is the Frobenius inner product. The $hI_d$ part of $\Delta W_i\Delta W_i^{\top}$ accounts for the Brownian Laplacian drift, while $Q_i$ isolates the centered Hessian fluctuation. Train with $\mathcal{L}_\theta=\frac{1}{N}\sum_{i=0}^{N-1}\mathbb{E}[R_i^2]+\lambda_g\mathbb{E}[\|\nabla_xU_\theta(T,X_T)-\nabla g(X_T)\|_2^2]+\lambda_y\mathbb{E}[|U_\theta(T,X_T)-g(X_T)|^2]$. Here $h=T/N$, $f$ is the known nonlinear PDE driver, $g$ is the terminal target, and $\lambda_y,\lambda_g\geq0$ are penalty weights.

Implementation notes

1. Integration point: use this as the training loss for a scalar MLP $U_\theta(t,x)$ in a high-dimensional parabolic PDE solver. Do not create separate networks for value, gradient, or Hessian; obtain $Y_i$, $Z_i$, and $\Gamma_i$ with automatic differentiation. The Hessian can be represented as a full matrix for small dimensions, or computed through Hessian-vector products when the dimension is large. 2. Pseudocode: sample $X_0$ from the selected initial distribution. For every trajectory and every grid index $i$, sample $\Delta W_i\sim\mathcal{N}(0,hI_d)$, set $X_{i+1}=X_i+\Delta W_i$, compute $Y_i=U_\theta(t_i,X_i)$, $Z_i=\nabla_xU_\theta(t_i,X_i)$, and $\Gamma_i=D_x^2U_\theta(t_i,X_i)$. Compute $Y_{i+1}=U_\theta(t_{i+1},X_{i+1})$ and $Q_i=\Delta W_i\Delta W_i^\top-hI_d$. Evaluate $f(t_i,X_i,Y_i,Z_i,\Gamma_i)$, form $R_i$ from the displayed formula, average $R_i^2$ over trajectories and time steps, add the terminal value and gradient penalties, backpropagate, and update $\theta$ with AdamW. 3. The mathematics determines the Brownian transition, centered quadratic probe, and residual; expectations are estimated by minibatch averages. Compute $\nabla g$ analytically for manufactured problems or by autodifferentiating the known terminal function. Log separate value, gradient, Hessian, terminal, and residual losses. 4. First experiment: solve a 20- or 50-dimensional manufactured PDE with known smooth solution, comparing this objective against a standard pointwise PINN residual and a first-order stochastic BSDE loss at equal network size and sampled function evaluations. Sweep $N\in\{10,20,40\}$ and measure value, gradient, and Hessian errors on held-out Brownian states. Success means lower full-jet error at equal wall-clock cost, decreasing error as $h$ decreases, and less instability when $f$ has moderate Hessian dependence.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a scalar-MLP Brownian second-order residual with autodifferentiated gradient/Hessian, terminal value and gradient penalties, plus a pointwise PINN comparator. The mechanism checks succeeded: the centered quadratic probe had observed h-scaling 0.99996 versus predicted 1.0 with maximum relative RMS error 0.19%, and the cubic Taylor remainder had observed h^1.5-scaling 1.49947 versus predicted 1.5 with 0.42% maximum relative error. The small training comparison did not show an accuracy or speed win for the Brownian method: its errors were generally worse and it took about 19.3 seconds versus 4.0 seconds for the pointwise baseline, so the result supports the mathematical probe mechanism but not superiority in this toy setup.

Agent confidence
8/10
Baseline
Pointwise PINN: value MAE 0.1183, gradient MAE 0.2192, Hessian MAE 0.2295, runtime 4.06 s; with terminal gradient penalty: 0.1226 / 0.2148 / 0.1876, 3.96 s.
Idea
Brownian jet residual: value MAE 0.2315, gradient MAE 0.2910, Hessian MAE 0.2561, runtime 19.32 s; with terminal gradient penalty: 0.2375 / 0.2620 / 0.2175, 19.27 s. Probe RMS relative error was <=0.19%; cubic remainder RMS relative error was <=0.42%.

Limitations: Only a one-dimensional manufactured PDE was trained, with a small width-32 network, batch size 32, N=8, and 280 optimization steps. No 20/50/100-dimensional benchmark, Hessian-dependent nonlinear driver, N sweep, equal-FLOP normalization, variance study, or larger terminal-gradient ablation was tested.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.16730)

Significantly WORSE than the tuned baseline (+40.1%, p=0.0149, wins 1 / 8 paired seeds).

Benchmark
Manufactured_heat_pde
Model
mlp_tiny
Paired seeds
8
Baseline mean
0.0606
Idea mean
0.0848
Effect (Δ)
0.0243 (+40.1%; negative = idea better)
Wins
1 / 8 paired seeds
p-value
0.0149 (permutation test, 20 000 shuffles)
Smallest detectable effect
±25.0%
Mechanism
Not confirmed ✗
Practical effect
Hurts
Baseline tuning
swept over 3 configs
Limitations:

Only the required custom 1D manufactured PDE track was tested because no built-in benchmark has PDE structure. The shared bench mlp_tiny uses ReLU, making its autodifferentiated Hessian identically zero; no smooth-activation architecture or higher-dimensional PDE was evaluated.

How to run: python3 bench_experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark latest Failed ✗

Agent confidence: 10/10

Built and ran the mandatory 8-seed paired benchmark on a structurally matched custom manufactured parabolic PDE track, using the same mlp_tiny architecture and shared learning-rate union for baseline and idea. The Brownian second-order residual was significantly worse: MSE 0.0848281 versus 0.0605581 for the pointwise PINN, paired delta +0.0242700 with permutation p=0.01485. The trained ReLU MLP had zero Hessian, so the NN-scale quadratic-probe mechanism signature was not confirmed.

Baseline
Pointwise PDE residual, best lr=0.01: mean test MSE 0.060558140743523836, std 0.013661196266041849.
Idea
Brownian second-order residual, best lr=0.01: mean test MSE 0.08482814487069845, std 0.015319140730036727.
Idea wins
1
N pairs
8
Paired delta mean
0.024270004127174616
Permutation p value
0.01485

Benchmark result

Significantly WORSE than the tuned baseline (+40.1%, p=0.0149, wins 1 / 8 paired seeds).

Benchmark
Manufactured_heat_pde
Model
mlp_tiny
Paired seeds
8
Baseline mean
0.0606
Idea mean
0.0848
Effect (Δ)
0.0243 (+40.1%; negative = idea better)
Wins
1 / 8 paired seeds
p-value
0.0149 (permutation test, 20 000 shuffles)
Smallest detectable effect
±25.0%
Mechanism
Not confirmed ✗
Practical effect
Hurts
Baseline tuning
swept over 3 configs
Limitations:

Only the required custom 1D manufactured PDE track was tested because no built-in benchmark has PDE structure. The shared bench mlp_tiny uses ReLU, making its autodifferentiated Hessian identically zero; no smooth-activation architecture or higher-dimensional PDE was evaluated.

How to run: python3 bench_experiment.py

Artifacts

Implementation overview ⬇ Download all as ZIP 7 files · code, reports and structured results