ReLU$^k$ Neural de Rham Complexes

arXiv:2607.22478 2026 Architecture 1 ideas extracted · analyzed Aug 30, 2026

What the math gives to ML

The paper provides a finite-dimensional neural analogue of the de Rham complex: fixed-neuron ReLU ridge functions are assigned to differential-form degrees so that the exterior derivative is represented exactly inside the network space. The key transferable asset is structural rather than a new scalar activation: differentiation lowers the ReLU power and becomes a fixed wedge operation, yielding neuron-wise Koszul complexes, exactness, and therefore no spurious closed-but-non-exact modes under a linear-independence condition. A practical use is a multi-form neural architecture for PDEs and geometric learning in which potentials, vector fields, curls, and divergences are connected by hard algebraic maps instead of soft penalties.

Ideas from this paper

Mechanism confirmed, baseline not beaten 2026

Exact Neural de Rham Backbone

Replace independently parameterized scalar, vector, and higher-order neural outputs with consecutive spaces of ReLU-power differential forms linked by an exact exterior-derivative layer. The network can then produce curl-free, divergence-free, or more general closed fields by construction, while the complex prevents artificial null-space modes that commonly appear when differential constraints are enforced only through sampled residual losses.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: ReLU$^k$ Neural de Rham Complexes arXiv:2607.22478