# Gram-Whitened Directional Pooling

- ID: 2207
- Canonical URL: https://synthcore.org/idea/2207/gram-whitened-directional-pooling
- API JSON: https://synthcore.org/api/idea/2207.json
- API Markdown: https://synthcore.org/api/idea/2207.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.12794](https://arxiv.org/abs/2608.12794)
- Category: architecture
- Solves: accuracy, stability, generalization
- ML areas: embedding, cnn, transformer, loss
- Math tags: functional-analysis, linear-algebra, geometry, approximation-theory
- Ratings: usefulness 7/10; difficulty 5/10; novelty 6/10

## Idea description

Represent local feature channels as a smooth directional signal and aggregate them with a partition-of-unity family of learnable spherical atoms instead of hard angular bins. Use the atom Gram matrix to whiten the descriptor and add a projected-energy loss, so the network is rewarded for retaining information in the directional subspace rather than merely producing large correlated channel responses.

## Mathematical statement

The descriptor has coordinates \(\phi_i(f)=\langle f,w_i\rangle_{L^2(\mathbb S^{d-1})}=\int_{\mathbb S^{d-1}}f(\theta)w_i(\theta)\,d\sigma(\theta)\), where \(f\) is a directional feature signal, \(w_i\) is the \(i\)-th nonnegative weight, and \(\sigma\) is surface measure. The weights satisfy the partition of unity \(\sum_{i=1}^n w_i(\theta)=1\), \(w_i(\theta)\ge 0\). For unit directions \(d_i\in\mathbb S^{d-1}\) and temperature \(\varepsilon>0\), use \(w_i(\theta)=\exp(d_i^\top\theta/\varepsilon)/\sum_{j=1}^n\exp(d_j^\top\theta/\varepsilon)\). Define the Gram matrix \(G\in\mathbb R^{n\times n}\) by \(G_{ij}=\langle w_i,w_j\rangle_{L^2}=\int w_i(\theta)w_j(\theta)\,d\sigma(\theta)\), and descriptor vector \(c_i=\phi_i(f)\). The orthogonal projection onto \(W_n=\operatorname{span}\{w_i\}\) is \(P_nf(\theta)=\sum_{i,j=1}^n w_i(\theta)(G^\dagger)_{ij}c_j\), where \(G^\dagger\) is the Moore-Penrose pseudoinverse. Its induced seminorm is \(|f|_{\mathcal D,n}^2=\|P_nf\|_{L^2}^2=c^\top G^\dagger c\), and this converges to \(\|f\|_{L^2(\mathbb S^{d-1})}^2\) as the directional family becomes total. In a network, this gives a principled decorrelated descriptor and a loss measuring retained directional energy.

## Key formulas

- $$\phi_i(f)=\langle f,w_i\rangle_{L^2(\mathbb S^{d-1})}=\int_{\mathbb S^{d-1}}f(\theta)w_i(\theta)\,d\sigma(\theta),\qquad \sum_{i=1}^{n}w_i(\theta)=1,\quad w_i(\theta)\geq 0.$$
- $$w_i(\theta)=\frac{\exp(d_i^\top\theta/\varepsilon)}{\sum_{j=1}^{n}\exp(d_j^\top\theta/\varepsilon)},\qquad d_i\in\mathbb S^{d-1},\ \varepsilon>0.$$
- $$G_{ij}=\langle w_i,w_j\rangle_{L^2}=\int_{\mathbb S^{d-1}}w_i(\theta)w_j(\theta)\,d\sigma(\theta),\qquad P_nf(\theta)=\sum_{i,j=1}^{n}w_i(\theta)(G^\dagger)_{ij}\phi_j(f).$$
- $$|f|_{\mathcal D,n}^{2}=\|P_nf\|_{L^2}^{2}=\phi(f)^\top G^\dagger\phi(f),\qquad |f|_{\mathcal D,n}\longrightarrow\|f\|_{L^2(\mathbb S^{d-1})}\ \text{as }n\to\infty.$$

## Implementation notes

Integrate this into a CNN or vision-transformer block immediately after a local feature tensor has been converted into directional samples. For each spatial token x, construct K unit directions theta_k from fixed image-plane offsets, 3D point-neighborhood offsets, or a learned normalized direction head. Let F_{x,k} in R^C be the feature at direction theta_k. Maintain n learnable unit atoms d_i, and compute W_{k,i}=softmax_i(d_i^T theta_k/epsilon). Aggregate descriptor channels as c_{x,i}=sum_k q_k F_{x,k} W_{k,i}, where q_k are quadrature weights summing to one. Precompute or update the Gram matrix G_{ij}=sum_k q_k W_{k,i}W_{k,j}. Stabilize it with G_lambda=G+lambda I, solve G_lambda z_x=c_x by Cholesky, and feed the whitened descriptor z_x to the next layer; stop gradients through G initially, then optionally update it with an exponential moving average. Add L_energy=||c_x||_2^2-c_x^T G_lambda^{-1}c_x only if the target task benefits from preserving directional energy, or use c_x^T G_lambda^{-1}c_x as an auxiliary representation norm. The paper supplies the projection and Gram formulas; the engineer estimates integrals by the finite directional quadrature above and chooses lambda empirically. First test on CIFAR-100 or ModelNet40 using a small ResNet with four angular atoms versus hard directional binning and ordinary softmax pooling. Measure validation accuracy, sensitivity to rotated inputs, atom-correlation condition number, and training-loss variance. Success means better rotation robustness or accuracy at equal FLOPs, with a lower Gram condition number and no instability when epsilon is annealed.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: no_effect
- Verdict: Built a NumPy MVP for softmax directional atoms, finite-quadrature Gram matrices, Gram whitening, projected energy, and ridge stabilization. The checks confirmed partition-of-unity error 3.3e-16, projection energy 0.2967 <= signal energy 0.845, projection reconstruction identity error 9.4e-16, monotone energy reduction across lambda, and asymptotic lambda*E=0.0342766 versus predicted c^T c=0.0342827. The dense atom sweep approached full signal energy (0.8450), validating the totality-style trend, but the toy classification accuracy was identical for raw pooling and whitening (0.5854), so no downstream performance win was observed.

### Mechanism check

- Verdict: Built a NumPy MVP for softmax directional atoms, finite-quadrature Gram matrices, Gram whitening, projected energy, and ridge stabilization. The checks confirmed partition-of-unity error 3.3e-16, projection energy 0.2967 <= signal energy 0.845, projection reconstruction identity error 9.4e-16, monotone energy reduction across lambda, and asymptotic lambda*E=0.0342766 versus predicted c^T c=0.0342827. The dense atom sweep approached full signal energy (0.8450), validating the totality-style trend, but the toy classification accuracy was identical for raw pooling and whitening (0.5854), so no downstream performance win was observed.
- Confidence: 8/10
- Limitations: Only a small synthetic 2D directional signal experiment was tested; no CNN, CIFAR-100/ModelNet40, learned atoms, rotation-robustness benchmark, GPU execution, FLOP measurement, or training-loss variance study was performed. The atom-count energy trend was sampled and approached full energy but was not strictly monotone at every intermediate count.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.8129
- Paired wins: 4/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_report.json](https://synthcore.org/code/878/bench_report.json)
- [gram_directional_pooling.py](https://synthcore.org/code/878/gram_directional_pooling.py)
- [report.md](https://synthcore.org/code/878/report.md)
- [report_bench_2026-09-01T133239.md](https://synthcore.org/code/878/report_bench_2026-09-01T133239.md)
- [results.json](https://synthcore.org/code/878/results.json)
- [stage2_bench.py](https://synthcore.org/code/878/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/878)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.