Binary Very-Weak PDE Network
Implementation & benchmark of arXiv:2607.14498 — Neural Very Weak Formulations enabling Hardware-Oriented deep PDE solvers
Source paper: Neural Very Weak Formulations enabling Hardware-Oriented deep PDE solvers arXiv:2607.14498 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Combine the very-weak residual with step activations and one-bit weights, so the deployed PDE solver uses threshold and binary operations while training still optimizes a differentiable surrogate. The weak objective only needs values of the trial function and therefore does not require differentiating discontinuous activations with respect to spatial coordinates.
Formulas
Mathematical statement
The construction uses the very-weak identity $-\int_\Omega u\Delta w=-\int_\Gamma g\partial_\eta w+\int_\Omega fw$ for smooth $w\in H^2(\Omega)\cap H_0^1(\Omega)$. The trial network can therefore use the step activation $H(z)$, defined as $1$ for $z\geq 0$ and $0$ otherwise, without needing classical spatial derivatives of $H$. For one-bit weights use $q(a)=\operatorname{sign}(a)\in\{-1,+1\}$. During training, replace $H$ by the smooth surrogate $H_\tau(z)=\sigma(z/\tau)$ or use a straight-through estimator, where $\tau>0$ is an annealing temperature. The residual is $r_j=-\int_\Omega u_\theta\Delta w_j+\int_\Gamma g\partial_\eta w_j-\int_\Omega fw_j$. A practical quantization penalty is $\beta\|\theta-q(\theta)\|_2^2$, encouraging floating-point parameters to approach their one-bit projections.
Implementation notes
Integrate this at both the trial-network architecture and PDE loss. Start with a boundary-preserving form $u_\theta(x)=\ell(x)+d(x)F_\theta(x)$, where $d(x)=0$ on the boundary. Build $F_\theta$ from binary linear layers and step activations. Represent each weight matrix as $W=s_W\operatorname{sign}(A_W)$, where $A_W$ is a trainable real-valued matrix and $s_W>0$ is a learned per-layer or per-output-channel scale. During training use $H_\tau(z)=\sigma(z/\tau)$ in the backward computation and anneal $\tau$ from $1$ to $0.05$; alternatively use hard $H$ in the forward pass with the surrogate derivative $H_\tau'(z)$ in the backward pass. For each batch of quadrature points, evaluate $u_\theta$ and compute r[j] = -sum_q(a[q] * u[q] * lap_w[j,q]) + boundary_term[j] - forcing_term[j]. Optimize the squared residual plus beta * mean((theta - sign(theta))**2) or its scale-aware version. At the end, replace every activation by hard $H$ and every weight by its sign times its learned scale, then reevaluate the residual and PDE error without retraining. The weak residual is taken from the paper; temperature annealing, scale placement, and the quantization penalty are empirical. First benchmark a 2D unit-square Poisson problem with two to four binary layers against a float MLP with the same parameter count. Also compare with a strong-form binary PINN if stable. Record parameter bytes, CPU/GPU latency, relative $L^2$ error, weak residual, and error change after hardening. Success means at least 8x parameter-storage reduction and materially lower inference latency with less than 2x error increase.
Verification
Stage 1 · Toy mechanism gate: Failed ✗
Stage 2 · Mechanism transferred to benchmark: Not tested
Stage 2 · Practical benchmark result: Not run
Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built a boundary-preserving binary/step very-weak PDE network with sigmoid training surrogate, straight-through signed weights, quantization penalty, hard deployment evaluation, and analytic test-function Laplacians. The math checks confirmed midpoint-quadrature refinement ratios of 4.49, 4.12, and 4.03 versus the predicted ratio 4, and the bubble factor gave exactly zero hard output on sampled boundary points; lowering temperature also reduced the soft-to-hard output gap from 0.00614 to 0.00095. However, the binary model had relative L2 error 0.946 versus 0.530 for float, weak residual RMS 0.0118 versus 0.000435, and was 6.3x slower in PyTorch inference, despite a 20.4x estimated parameter-storage reduction, so this is not a demonstrated deployment win.
- Agent confidence
- 9/10
- Baseline
- Float MLP: relative_L2=0.5299, weak_residual_rms=0.0004349, inference=1.009 ms/batch of 5000, storage=1348 bytes
- Idea
- Binary step network: relative_L2=0.9464 (1.79x baseline), hard weak_residual_rms=0.01178, inference=6.363 ms/batch of 5000 (0.16x baseline speed), estimated storage=66.1 bytes (20.4x reduction), hardening output gap at tau=0.005=0.000952
Limitations: Only a small 2D unit-square smooth Poisson problem was tested, with one hidden width and one seed. Binary operations were simulated by ordinary PyTorch kernels rather than a packed-bit accelerator, so hardware latency and energy benefits were not measured. The benchmark did not test stronger baselines, larger networks, nonzero boundary data, singular forcing, or temperature annealing during training.
How to run: python3 binary_vwf_experiment.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Artifacts
- binary_vwf_experiment.py 9.0 KB View Raw
- report.md 1.8 KB View
- results.json 3.3 KB View Raw