△ Mechanism confirmed, baseline not beaten 2026

Exact Neural de Rham Backbone

Implementation & benchmark of arXiv:2607.22478 — ReLU$^k$ Neural de Rham Complexes

Usefulness7/10

Difficulty5/10

Novelty7/10

Architecture Accuracy Stability Sample efficiency Mlp Graph nn World model Differential geometry Topology Linear algebra Approximation theory

Source paper: ReLU$^k$ Neural de Rham Complexes arXiv:2607.22478 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace independently parameterized scalar, vector, and higher-order neural outputs with consecutive spaces of ReLU-power differential forms linked by an exact exterior-derivative layer. The network can then produce curl-free, divergence-free, or more general closed fields by construction, while the complex prevents artificial null-space modes that commonly appear when differential constraints are enforced only through sampled residual losses.

Formulas

$$s_i(x)=\omega_i\cdot x+b_i,\qquad \sigma_m(t)=\frac{(t_+)^m}{m!},\qquad ds_i=\sum_{r=1}^{d}\omega_{ir}\,dx_r.$$

$$d\!\left(\sigma_{k-p}(s_i)\alpha_i\right)=\sigma_{k-p-1}(s_i)\,ds_i\wedge\alpha_i,$$

$$K_{p+1}^{(i)}K_p^{(i)}\alpha=ds_i\wedge(ds_i\wedge\alpha)=0,$$

$$\Omega^p_{\mathrm{NN}}=\left\{\omega_p=\sum_{i=1}^{n}\sigma_{k-p}(s_i)\alpha_i\;:\;\alpha_i\in\Lambda^p(\mathbb{R}^d)\right\},\qquad d\omega_p=\sum_{i=1}^{n}\sigma_{k-p-1}(s_i)\bigl(ds_i\wedge\alpha_i\bigr).$$

Mathematical statement

For a neuron affine function s_i(x)=\omega_i\cdot x+b_i, with \omega_i\in\mathbb{R}^d and b_i\in\mathbb{R}, define the normalized ReLU power \sigma_m(t)=(t_+)^m/m! where t_+=\max(t,0), and the constant one-form ds_i=\sum_{r=1}^d\omega_{ir}\,dx_r. The neural p-form space uses coefficients \sigma_{k-p}(s_i): \Omega^p_{\mathrm{NN}}=\{\sum_i\sigma_{k-p}(s_i)\alpha_i:\alpha_i\in\Lambda^p(\mathbb{R}^d)\}. The essential identity is d\sigma_m(s_i)=\sigma_{m-1}(s_i)\,ds_i, so the exterior derivative maps \Omega^p_{\mathrm{NN}} to \Omega^{p+1}_{\mathrm{NN}} by d(\sigma_{k-p}(s_i)\alpha_i)=\sigma_{k-p-1}(s_i)(ds_i\wedge\alpha_i). For each neuron, the resulting maps are Koszul differentials K_p^{(i)}\alpha=ds_i\wedge\alpha. Since ds_i\wedge ds_i=0, consecutive maps compose to zero. Under linear independence of the lowest-order family \{\sigma_{k-d}(s_i)\}_{i=1}^n, the global complex decomposes neuron-wise; exactness can therefore be checked or enforced by the rank of these small Koszul maps rather than by PDE collocation alone.

Implementation notes

Integrate the construction as a shared fixed-feature backbone for a PDE or geometric-learning model. Choose dimension d, ReLU order k\ge d, and n fixed neurons (\omega_i,b_i); initialize \omega_i quasi-uniformly on the unit sphere and freeze them. Given a batch of coordinates x, compute S[i,b]=\omega_i\cdot x_b+b_i and F[m,i,b]=max(S[i,b],0)^m/m! for m=0,...,k. Store each p-form coefficient as a tensor C[p,i,basis,b], where basis indexes the \binom{d}{p} coordinate wedges. For each p, apply an explicit sparse exterior-wedge matrix W_i[p] induced by ds_i: D[p+1,i,:,:]=F[k-p-1,i,:]*W_i[p]@C[p,i,:,:]. Equivalently: S=x@omega.T+b; F[m]=relu(S)**m/factorial(m); for p in range(k): for i: form[p+1,i]=F[k-p-1,i]*wedge(ds[i], form[p,i]). Do not estimate derivatives with finite differences; use this formula directly or verify it with automatic differentiation. The paper supplies the algebraic identity d^2=0 and the neuron-wise Koszul structure. Empirically estimate conditioning by forming the sampled feature matrix A[b,i]=F[k-d,i,b] and monitoring its smallest singular value; reject or resample neuron sets with near-zero singular values. First test on a 2D unit-square Poisson problem and a steady incompressible-flow surrogate: compare an unconstrained MLP, a potential-based baseline, and this architecture at equal parameter count. Measure PDE residual, boundary error, divergence/curl violation, smallest singular value of the discrete operator, and the number of spurious near-zero eigenmodes. A successful result is lower constraint violation at equal training steps, better conditioning, and faster convergence of the lowest PDE eigenvalues without increasing quadrature resolution.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built an explicit ReLU-power exact de Rham backbone with exterior-wedge maps and autodiff verification. The mechanism manifested: Koszul compositions were exactly zero, the derivative identity had max error 0, and the operator norm scaled linearly with neuron scale (observed log-log exponent 1.0 versus predicted 1.0). The curl-free constraint was exact (4.5e-16 RMS), but the constrained model had much worse vector-fitting MSE than the larger unconstrained MLP, so this is a structural win rather than an accuracy win in this toy comparison.

Agent confidence: 9/10

Baseline: Unconstrained 722-parameter MLP: MSE 0.001458, curl RMS 0.71645
Idea: 28-coefficient exact backbone: MSE 0.259014, curl RMS 4.54e-16; nilpotence max error 0, derivative identity max error 0, Koszul scaling exponent 1.0

Limitations: The PDE comparison was a small supervised vector-field surrogate rather than a full Poisson or incompressible-flow solve. Parameter counts were not matched (28 versus 722), no conditioning/eigenmode study or feature resampling was performed, and only the 0-form to 1-form exact-gradient case was trained.

How to run: python3 exact_derham_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.22478) Sep 2, 2026

Significantly WORSE than the tuned baseline (+240.5%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Poisson_boundary
Model: scalar_potential_backbone
Paired seeds: 8
Baseline mean: 0.0109
Idea mean: 0.0372
Effect (Δ): 0.0263 (+240.5%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±106.4%
Mechanism: Confirmed ✓
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

The registered task is supervised Poisson boundary-value regression rather than explicit PDE collocation with separate interior and boundary residual losses. Higher-form divergence-free experiments, conditioning/eigenmode counts, broader feature-count sweeps, and larger datasets were not tested.

How to run: python3 run_bench_registered.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 10/10

The registered PDE benchmark completed successfully on `poisson_boundary` with 8 paired seeds. The exact ReLU-power potential mechanism was confirmed by trained-model behavior (observed curl RMS 1.15e-7 versus predicted 0), but the idea was significantly worse on the independent task metric: MSE delta +0.02630 and permutation p=0.0081.

Baseline: Best lr=0.03; test MSE mean 0.0109364952
Idea: Best lr=0.03; test MSE mean 0.0372408738; paired delta +0.0263043786; permutation p=0.0081

Benchmark result