# Parity-Aware Projective Feature Layer

- ID: 276
- Canonical URL: https://synthcore.org/idea/276/parity-aware-projective-feature-layer
- API JSON: https://synthcore.org/api/idea/276.json
- API Markdown: https://synthcore.org/api/idea/276.md
- Verification status: unverified
- Source: [arXiv:2606.31429](https://arxiv.org/abs/2606.31429)
- Category: architecture
- Solves: memory, stability, accuracy
- ML areas: mlp, embedding, quantization, initialization
- Math tags: geometry, probability, functional-analysis, statistics
- Ratings: usefulness 5/10; difficulty 3/10; novelty 7/10

## Idea description

Exploit the paper's distinction between spherical and projective feature geometry by tying antipodal neurons whenever the activation and task have the corresponding parity. This removes duplicate particles representing the same projective direction and makes the learned feature distribution explicitly even or odd.

## Mathematical statement

The paper defines the learned feature map by \(\varphi_{\mathrm{neur}}(x)(w)=\sigma(\langle w,x\rangle)\) and the induced feature kernel by \(K_{\mathrm{feat}}(x_1,x_2)=\mathbb{E}_{W\sim\hat\varphi_\lambda}[\sigma(\langle W,x_1\rangle)\sigma(\langle W,x_2\rangle)]\), where \(\hat\varphi_\lambda\) is the stationary marginal of feature parameters. For an even activation, \(\sigma(-u)=\sigma(u)\), the directions \(w\) and \(-w\) are identical and naturally live on real projective space \(\mathbb{RP}^{d-1}\); for an odd activation, \(\sigma(-u)=-\sigma(u)\), antipodal features can be tied with opposite output coefficients. Enforce this algebraically with paired particles \((a_i,w_i)\) and \((s a_i,-w_i)\), where \(s=+1\) for even and \(s=-1\) for odd parity. The paired contribution is \(a_i[\sigma(w_i^\top x)+s\sigma(-w_i^\top x)]/M\), so redundant antipodal sampling is removed while the kernel remains parity-consistent.

## Key formulas

- $$\varphi_{\mathrm{neur}}(x)(w)=\sigma(\langle w,x\rangle)$$
- $$K_{\mathrm{feat}}(x_1,x_2)=\mathbb{E}_{W\sim\hat{\varphi}_{\lambda}}\left[\sigma(\langle W,x_1\rangle)\sigma(\langle W,x_2\rangle)\right]$$
- $$\sigma(-u)=s\,\sigma(u),\quad s\in\{+1,-1\}$$
- $$h(x)=\frac{1}{M}\sum_{i=1}^{M}a_i\left[\sigma(w_i^\top x)+s\sigma((-w_i)^\top x)\right]$$

## Implementation notes

Use this in a feature layer whose weights are constrained to unit-norm directions, especially cosine MLPs, random-feature regressors, or attention key/query projections when inputs are directional. Store only \(M\) representative directions \(w_i\) and coefficients \(a_i\); generate the antipodal partner on the fly rather than storing a second independent neuron. Set \(s=+1\) for an even activation such as absolute value or a deliberately symmetrized activation, and \(s=-1\) for an odd activation such as tanh. For each minibatch, compute both dot products \(w_i^\top x\) and \(-w_i^\top x\), combine them using the paired formula, backpropagate through the representative parameters only, and renormalize \(w_i\leftarrow w_i/\max(\|w_i\|,\epsilon)\). Estimate the induced kernel empirically on a validation subset using \(\widehat K(x,x')=M^{-1}\sum_i\phi_i(x)\phi_i(x')\), and compare its spectrum and condition number with an untied layer. The first cheap experiment is Gaussian single-index regression with even and odd teachers, comparing an untied width-\(2M\) network, a tied width-\(M\) network, and a standard width-\(M\) network. Success means approximately half the feature parameters and lower kernel redundancy at matched test MSE, together with improved stability across random seeds; failure is no accuracy gain or a substantial increase in optimization time from evaluating paired activations.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.