# Null-space conservation projection

- ID: 826
- Canonical URL: https://synthcore.org/idea/826/null-space-conservation-projection
- API JSON: https://synthcore.org/api/idea/826.json
- API Markdown: https://synthcore.org/api/idea/826.md
- Verification status: beats_baseline
- Source: [arXiv:2607.12759](https://arxiv.org/abs/2607.12759)
- Category: optimization
- Solves: stability, accuracy, generalization
- ML areas: loss, optimizer, world-model, training-dynamics
- Math tags: optimization, linear-algebra, numerical-analysis
- Ratings: usefulness 7/10; difficulty 5/10; novelty 5/10

## Idea description

Add an exact linear-constraint projection to the output solve of a neural operator or physics-informed model. The network produces an unconstrained prediction or coefficient vector, while a small constrained least-squares layer removes the component violating known conservation laws and separately penalizes residuals that cannot be enforced exactly.

## Mathematical statement

The paper solves a weighted constrained least-squares problem, $$\min_{\vec q\in\mathbb R^{n_{unk}}}\{\|A_1\vec q-\vec b_1\|_2^2+\lambda_D W_D^2\|D\vec q\|_2^2\ \mid\ C\vec q=\vec 0\},$$ where $\vec q$ is the unknown coefficient vector, $A_1\vec q\approx\vec b_1$ are the primary interface equations, $D\vec q$ is a secondary flux-balance residual, $\lambda_D\geq0$ is its penalty weight, $W_D$ rescales that residual, and $C\vec q=0$ contains one exact area constraint per bounded phase. Let $H=A_1^    op A_1+\lambda_DW_D^2D^    op D+\epsilon I$ and $g=A_1^    op b_1$, with $\epsilon>0$ for conditioning. The unconstrained solution is $q_0=H^{-1}g$. Its minimum-norm correction is $$q=q_0-C^\top(CC^\top+\epsilon_CI)^{-1}Cq_0,$$ which gives $Cq\approx0$. Equivalently, solve the KKT system $$\begin{bmatrix}H&C^\top\\C&0\end{bmatrix}\begin{bmatrix}q\\\mu\end{bmatrix}=\begin{bmatrix}g\\0\end{bmatrix}.$$ Here $C\in\mathbb R^{m    imes n}$ maps coefficients or grid values to $m$ conserved quantities. For a mass-conserving field with quadrature weights $\omega_j$, a constraint row is $C_j=\omega_j$ and the target equation is $Cq=M_0$; subtracting a reference state converts it to the paper's homogeneous form.

## Key formulas

- $$\min_{\vec q\in\mathbb R^{n_{unk}}}\left\{\|A_1\vec q-\vec b_1\|_2^2+\lambda_D W_D^2\|D\vec q\|_2^2\ \middle|\ C\vec q=\vec 0\right\}$$
- $$H=A_1^\top A_1+\lambda_DW_D^2D^\top D+\epsilon I,\qquad q_0=H^{-1}A_1^\top b_1$$
- $$q=q_0-C^\top(CC^\top+\epsilon_CI)^{-1}Cq_0$$
- $$\begin{bmatrix}H&C^\top\\C&0\end{bmatrix}\begin{bmatrix}q\\\mu\end{bmatrix}=\begin{bmatrix}A_1^\top b_1\\0\end{bmatrix}$$

## Implementation notes

(1) Integration point: use this as a differentiable output layer in a neural operator, PINN, or graph neural simulator. Let the network emit coefficients $z_\theta$ for a basis expansion $u(x)=\Phi(x)z_\theta$, or directly emit a grid field. Build $C$ from known quadrature weights, stoichiometric balances, total probability, divergence-free flux constraints, or per-component conserved masses. Put the main PDE or data residual in $A_1$ and auxiliary consistency residuals in $D$; do not enforce those secondary rows exactly. (2) Pseudocode: compute network output $z$; assemble or cache $A_1,D,C,b_1$; set $H=A_1^TA_1+\lambda_DW_D^2D^TD+\epsilon I$ and $g=A_1^Tb_1$; solve $q_0=H^{-1}g$; then compute $q=q_0-C^T(CC^T+\epsilon_CI)^{-1}(Cq_0-M)$ for target invariant vector $M$; decode $u=\Phi q$; backpropagate through the linear solve. Use QR or SVD for the small constraint system instead of explicitly forming an inverse. (3) Compute $C$ analytically from the conservation law and quadrature. Estimate conditioning empirically and tune $\lambda_D$; monitor $\|Cq-M\|$ as the hard-invariant metric. (4) First experiment: train a small Fourier neural operator or PINN on a 2-D advection-diffusion or Cahn-Hilliard dataset with known total mass. Compare ordinary residual training, a soft conservation penalty, and the projected layer at equal parameter count and FLOPs. Measure relative mass drift over long rollouts, validation error, gradient explosions, and wall-clock overhead. Success means mass error near solver precision, lower long-horizon rollout error, and improved stability without a large per-step cost increase.

## Verification

- Status: beats_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: helps
- Verdict: Built a differentiable null-space conservation projection layer and reproducible toy experiment in experiment.py. The mechanism predictions were confirmed: constraint residuals were about 1e-14, homogeneous correction scaling error was 0, and null-space pass-through error was below 5e-16. In the mini-experiment, projection reduced mean mass error to 6.7e-8 and validation MSE to 7.1e-5 versus 2.4e-2 mass error and 1.43e-4 MSE for the unconstrained baseline, at about 1.59x wall-clock time. A key caveat is that the supplied C^T(CC^T)^-1 formula is Euclidean projection and differed substantially from the H-weighted KKT optimum in this test.

### Mechanism check

- Verdict: Built a differentiable null-space conservation projection layer and reproducible toy experiment in experiment.py. The mechanism predictions were confirmed: constraint residuals were about 1e-14, homogeneous correction scaling error was 0, and null-space pass-through error was below 5e-16. In the mini-experiment, projection reduced mean mass error to 6.7e-8 and validation MSE to 7.1e-5 versus 2.4e-2 mass error and 1.43e-4 MSE for the unconstrained baseline, at about 1.59x wall-clock time. A key caveat is that the supplied C^T(CC^T)^-1 formula is Euclidean projection and differed substantially from the H-weighted KKT optimum in this test.
- Confidence: 8/10
- Limitations: Only a small synthetic vector regression task was tested, not a neural operator, PINN, PDE rollout, GPU execution, multiple random seeds, or large-scale conditioning. The training comparison uses Euclidean target projection and does not implement the H-weighted KKT projection as the training output layer.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.01485
- Paired wins: 7/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/337/bench_report.json)
- [bench_run.py](https://synthcore.org/code/337/bench_run.py)
- [conservation_track.py](https://synthcore.org/code/337/conservation_track.py)
- [experiment.py](https://synthcore.org/code/337/experiment.py)
- [report.md](https://synthcore.org/code/337/report.md)
- [report_bench_2026-08-31T012908.md](https://synthcore.org/code/337/report_bench_2026-08-31T012908.md)
- [results.json](https://synthcore.org/code/337/results.json)
- [Download all files as ZIP](https://synthcore.org/download/337)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.