# Convolution-Nullspace Feature Layer

- ID: 126
- Canonical URL: https://synthcore.org/idea/126/convolution-nullspace-feature-layer
- API JSON: https://synthcore.org/api/idea/126.json
- API Markdown: https://synthcore.org/api/idea/126.md
- Verification status: unverified
- Source: [arXiv:2608.24143](https://arxiv.org/abs/2608.24143)
- Category: architecture
- Solves: accuracy, stability, generalization
- ML areas: graph-nn, cnn, regularization
- Math tags: harmonic-analysis, functional-analysis, metric-geometry, linear-algebra
- Ratings: usefulness 5/10; difficulty 6/10; novelty 6/10

## Idea description

Construct a graph or sequence feature layer whose output satisfies a learned or fixed finite-support convolution equation, rather than allowing arbitrary features. For grid data, this is a hard local harmonicity or stencil constraint; for irregular graphs, it is imposed with a sparse incidence operator. The constraint can be exact through nullspace projection or soft through a residual penalty, and should improve robustness when the target signal is locally smooth or obeys known conservation relations.

## Mathematical statement

The paper describes translation-invariant subspaces on a group G through convolution equations of the form $(f*x_n)(g)=\sum_{h\in G}f(gh^{-1})x_n(h)=0$, where $f:G\to\mathbb{R}$ is a Lipschitz function, $x_n:G\to\mathbb{R}$ is a finite or summable constraint kernel, $g\in G$ is the translated location, and $*$ is group convolution. The associated constrained space is the set of Lipschitz functions annihilated by every translated kernel. The construction in the paper defines $T(x)=\sum_{n=1}^{\infty}x(n)f_n$ and controls its edge gradient using $|\nabla T(x)(e)|\leq\|x\|_{\ell_\infty}\sum_{n=0}^{\infty}2^{-2n}$, where $e$ is a graph edge and $\nabla f(e)$ is the endpoint difference across that edge. We adapt the first result to a finite stencil matrix $C$ and the second principle to a feature-Lipschitz regularizer: constrain $CH=0$ and penalize or normalize the edge-gradient norm. For a batch of node features $H\in\mathbb{R}^{N\times D}$, $C\in\mathbb{R}^{N\times N}$ is a sparse matrix assembled from translated local kernels; on a $d$-dimensional grid, $C$ is a convolution operator. The exact nullspace projection is $P=I-C^{\dagger}C$, where $C^{\dagger}$ is the Moore-Penrose pseudoinverse, so $H_{\mathrm{con}}=PH$ satisfies $CH_{\mathrm{con}}=0$ up to numerical error.

## Key formulas

- $$(f*x_n)(g)=\sum_{h\in G}f(gh^{-1})x_n(h)=0,$$
- $$T(x):=\sum_{n=1}^{\infty}x(n)f_n,$$
- $$|\nabla T(x)(e)|\leq\sum_{n=1}^{\infty}|x(n)\nabla f_n(e)|\leq\|x\|_{\ell_{\infty}}\sum_{n=0}^{\infty}2^{-2n}<\infty,$$
- $$H_{\mathrm{con}}=PH,\qquad P=I-C^{\dagger}C,\qquad CH_{\mathrm{con}}=0.$$

## Implementation notes

(1) Integration point: insert the module after a graph-convolution or Transformer token-mixing block and before the next nonlinear MLP. Let $H\in\mathbb{R}^{N\times D}$ be node or token features. Build a sparse local constraint matrix $C$ from a fixed stencil, such as the graph Laplacian $C=L$, a discrete biharmonic stencil, or a user-specified signed neighborhood kernel. For a grid, implement $CH$ as a depthwise convolution; for a graph, use sparse matrix multiplication. (2) Pseudocode: `H = mixer(X)`; `R = C @ H`; either use `Hc = H - lambda * C.T @ solve(C @ C.T + eps*I, R)` as an approximate projection, or precompute a low-rank/nullspace basis $U$ and use `Hc = U @ (U.T @ H)`; then `Y = mlp(Hc)`. Add `loss += alpha * mean((C @ Hc)**2)` if using a soft constraint. Estimate the edge-gradient statistic with `G_e = Hc[src]-Hc[dst]` and optionally add `beta * mean(norm(G_e,2))`. (3) Mathematics computed exactly: the local convolution residual $CH$, the projection formula, and the gradient-bound surrogate. Quantities such as the pseudoinverse, rank, and suitable $\alpha,\beta$ are estimated numerically; use conjugate gradients rather than forming $C^{\dagger}$ for large graphs. (4) First experiment: train a 4-layer GraphSAGE or GCN on Cora, Citeseer, and a noisy-grid image-classification task, comparing an unconstrained model, soft residual regularization, and projected features at equal parameter count. Measure validation accuracy under feature noise, the mean constraint residual $\|CH\|_F/\|H\|_F$, and gradient explosion during training. The idea succeeds if projection improves corruption accuracy or calibration without reducing clean accuracy by more than 1%, while keeping the residual near numerical zero and adding less than 20% wall-clock cost.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.