# Exact Neural de Rham Backbone

- ID: 1286
- Canonical URL: https://synthcore.org/idea/1286/exact-neural-de-rham-backbone
- API JSON: https://synthcore.org/api/idea/1286.json
- API Markdown: https://synthcore.org/api/idea/1286.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.22478](https://arxiv.org/abs/2607.22478)
- Category: architecture
- Solves: accuracy, stability, sample-efficiency
- ML areas: mlp, graph-nn, world-model
- Math tags: differential-geometry, topology, linear-algebra, approximation-theory
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Replace independently parameterized scalar, vector, and higher-order neural outputs with consecutive spaces of ReLU-power differential forms linked by an exact exterior-derivative layer. The network can then produce curl-free, divergence-free, or more general closed fields by construction, while the complex prevents artificial null-space modes that commonly appear when differential constraints are enforced only through sampled residual losses.

## Mathematical statement

For a neuron affine function s_i(x)=\omega_i\cdot x+b_i, with \omega_i\in\mathbb{R}^d and b_i\in\mathbb{R}, define the normalized ReLU power \sigma_m(t)=(t_+)^m/m! where t_+=\max(t,0), and the constant one-form ds_i=\sum_{r=1}^d\omega_{ir}\,dx_r. The neural p-form space uses coefficients \sigma_{k-p}(s_i): \Omega^p_{\mathrm{NN}}=\{\sum_i\sigma_{k-p}(s_i)\alpha_i:\alpha_i\in\Lambda^p(\mathbb{R}^d)\}. The essential identity is d\sigma_m(s_i)=\sigma_{m-1}(s_i)\,ds_i, so the exterior derivative maps \Omega^p_{\mathrm{NN}} to \Omega^{p+1}_{\mathrm{NN}} by d(\sigma_{k-p}(s_i)\alpha_i)=\sigma_{k-p-1}(s_i)(ds_i\wedge\alpha_i). For each neuron, the resulting maps are Koszul differentials K_p^{(i)}\alpha=ds_i\wedge\alpha. Since ds_i\wedge ds_i=0, consecutive maps compose to zero. Under linear independence of the lowest-order family \{\sigma_{k-d}(s_i)\}_{i=1}^n, the global complex decomposes neuron-wise; exactness can therefore be checked or enforced by the rank of these small Koszul maps rather than by PDE collocation alone.

## Key formulas

- $$s_i(x)=\omega_i\cdot x+b_i,\qquad \sigma_m(t)=\frac{(t_+)^m}{m!},\qquad ds_i=\sum_{r=1}^{d}\omega_{ir}\,dx_r.$$
- $$d\!\left(\sigma_{k-p}(s_i)\alpha_i\right)=\sigma_{k-p-1}(s_i)\,ds_i\wedge\alpha_i,$$
- $$K_{p+1}^{(i)}K_p^{(i)}\alpha=ds_i\wedge(ds_i\wedge\alpha)=0,$$
- $$\Omega^p_{\mathrm{NN}}=\left\{\omega_p=\sum_{i=1}^{n}\sigma_{k-p}(s_i)\alpha_i\;:\;\alpha_i\in\Lambda^p(\mathbb{R}^d)\right\},\qquad d\omega_p=\sum_{i=1}^{n}\sigma_{k-p-1}(s_i)\bigl(ds_i\wedge\alpha_i\bigr).$$

## Implementation notes

Integrate the construction as a shared fixed-feature backbone for a PDE or geometric-learning model. Choose dimension d, ReLU order k\ge d, and n fixed neurons (\omega_i,b_i); initialize \omega_i quasi-uniformly on the unit sphere and freeze them. Given a batch of coordinates x, compute S[i,b]=\omega_i\cdot x_b+b_i and F[m,i,b]=max(S[i,b],0)^m/m! for m=0,...,k. Store each p-form coefficient as a tensor C[p,i,basis,b], where basis indexes the \binom{d}{p} coordinate wedges. For each p, apply an explicit sparse exterior-wedge matrix W_i[p] induced by ds_i: D[p+1,i,:,:]=F[k-p-1,i,:]*W_i[p]@C[p,i,:,:]. Equivalently: `S=x@omega.T+b; F[m]=relu(S)**m/factorial(m); for p in range(k): for i: form[p+1,i]=F[k-p-1,i]*wedge(ds[i], form[p,i])`. Do not estimate derivatives with finite differences; use this formula directly or verify it with automatic differentiation. The paper supplies the algebraic identity d^2=0 and the neuron-wise Koszul structure. Empirically estimate conditioning by forming the sampled feature matrix A[b,i]=F[k-d,i,b] and monitoring its smallest singular value; reject or resample neuron sets with near-zero singular values. First test on a 2D unit-square Poisson problem and a steady incompressible-flow surrogate: compare an unconstrained MLP, a potential-based baseline, and this architecture at equal parameter count. Measure PDE residual, boundary error, divergence/curl violation, smallest singular value of the discrete operator, and the number of spurious near-zero eigenmodes. A successful result is lower constraint violation at equal training steps, better conditioning, and faster convergence of the lowest PDE eigenvalues without increasing quadrature resolution.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built an explicit ReLU-power exact de Rham backbone with exterior-wedge maps and autodiff verification. The mechanism manifested: Koszul compositions were exactly zero, the derivative identity had max error 0, and the operator norm scaled linearly with neuron scale (observed log-log exponent 1.0 versus predicted 1.0). The curl-free constraint was exact (4.5e-16 RMS), but the constrained model had much worse vector-fitting MSE than the larger unconstrained MLP, so this is a structural win rather than an accuracy win in this toy comparison.

### Mechanism check

- Verdict: Built an explicit ReLU-power exact de Rham backbone with exterior-wedge maps and autodiff verification. The mechanism manifested: Koszul compositions were exactly zero, the derivative identity had max error 0, and the operator norm scaled linearly with neuron scale (observed log-log exponent 1.0 versus predicted 1.0). The curl-free constraint was exact (4.5e-16 RMS), but the constrained model had much worse vector-fitting MSE than the larger unconstrained MLP, so this is a structural win rather than an accuracy win in this toy comparison.
- Confidence: 9/10
- Limitations: The PDE comparison was a small supervised vector-field surrogate rather than a full Poisson or incompressible-flow solve. Parameter counts were not matched (28 versus 722), no conditioning/eigenmode study or feature resampling was performed, and only the 0-form to 1-form exact-gradient case was trained.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/512/bench_report.json)
- [bench_report_registered.json](https://synthcore.org/code/512/bench_report_registered.json)
- [custom_exact_gradient_track.py](https://synthcore.org/code/512/custom_exact_gradient_track.py)
- [exact_derham_experiment.py](https://synthcore.org/code/512/exact_derham_experiment.py)
- [report.md](https://synthcore.org/code/512/report.md)
- [report_bench_2026-08-31T163351.md](https://synthcore.org/code/512/report_bench_2026-08-31T163351.md)
- [results.json](https://synthcore.org/code/512/results.json)
- [run_bench.py](https://synthcore.org/code/512/run_bench.py)
- [run_bench_registered.py](https://synthcore.org/code/512/run_bench_registered.py)
- [Download all files as ZIP](https://synthcore.org/download/512)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.