# Symplectic quadratic interaction layer

- ID: 191
- Canonical URL: https://synthcore.org/idea/191/symplectic-quadratic-interaction-layer
- API JSON: https://synthcore.org/api/idea/191.json
- API Markdown: https://synthcore.org/api/idea/191.md
- Verification status: unverified
- Source: [arXiv:2608.26001](https://arxiv.org/abs/2608.26001)
- Category: architecture
- Solves: memory, generalization, accuracy
- ML areas: embedding, graph-nn, mlp, regularization
- Math tags: algebra, representation-theory, combinatorics, geometry
- Ratings: usefulness 5/10; difficulty 5/10; novelty 7/10

## Idea description

Replace a dense second-order interaction layer on paired binary channels by a low-degree Boolean feature map obeying the quadratic-form cocycle from the paper. The layer uses only linear and pairwise features, but ties them through a learned or fixed symplectic form, reducing the number of independent interaction parameters and enforcing invariance under basis changes that preserve the pairing.

## Mathematical statement

The paper defines a symplectic vector space \(H=\mathbb{F}_2^{2g}\) with intersection form \(\widehat{\iota}:H\times H\to\mathbb{F}_2\), and a quadratic form \(q:H\to\mathbb{F}_2\) satisfying \(q(x+y)=q(x)+q(y)+\widehat{\iota}(x,y)\). For a coordinate symplectic basis \(S=\{a_1,b_1,\ldots,a_g,b_g\}\), the values \(q(a_i)=\lambda_i\) and \(q(b_i)=\lambda_i'\) determine \(q\) uniquely. The paper's Boolean algebra uses formal generators \(\overline{x}\) with the relation \(\overline{x+y}=\overline{x}+\overline{y}+\widehat{\iota}(x,y)\), and its degree-\(n\) subspace \(\operatorname{B}_n(g)\) has dimension \(\sum_{k=0}^n\binom{2g}{k}\); in particular \(\dim \operatorname{B}_2(g)=1+2g+\binom{2g}{2}\). We adapt this to real activations by pairing \(d=2g\) channels, choosing a skew-symmetric block matrix \(J\) with \(J_{a_i b_i}=1\), \(J_{b_i a_i}=-1\), and using the cocycle term \(x^\top J y\) to constrain second-order interactions. Over \(\mathbb{F}_2\), the quadratic relation is exact; over real activations, it becomes a parameter-sharing and augmentation-equivariance constraint.

## Key formulas

- $$q(x+y)=q(x)+q(y)+\widehat{\iota}(x,y),\qquad x,y\in H=\mathbb{F}_2^{2g}.$$
- $$\overline{x+y}=\overline{x}+\overline{y}+\widehat{\iota}(x,y).$$
- $$\dim \operatorname{B}_n(g)=\sum_{k=0}^{n}\binom{2g}{k},\qquad \dim \operatorname{B}_2(g)=1+2g+\frac{2g(2g-1)}{2}=2g^2+g+1.$$
- $$\mathcal{L}_{\mathrm{symp}}=\mathbb{E}_{U\sim\mathcal{G}}\left[\left\|F(Uz)-\rho(U)F(z)\right\|_2^2\right],\qquad U^\top J U=J.$$

## Implementation notes

Integrate this as a replacement for the first dense pairwise-interaction block in a small MLP or graph neural network. Let the incoming tensor be \(z\in\mathbb{R}^{B\times d}\), with \(d=2g\) channels grouped into \(g\) pairs. First compute a linear term \(u=W_1z\). For each pair \((a_i,b_i)\), compute the symplectic interaction \(s_i=z_{a_i}z_{b_i}\); optionally compute cross-pair terms only through tied coefficients determined by the fixed matrix \(J\), rather than learning an independent coefficient for every ordered pair. Concatenate \([1,z,s]\), apply normalization and a projection \(W_2\), and add the result residually to the original block. A practical pseudocode version is: `z = input; u = W1 @ z; s_i = z[:,a_i]*z[:,b_i] for i=1..g; phi = concat(ones(B,1), z, s); out = W2 @ phi + bias; y = z + gate*tanh(out)`. For a stronger version, binarize or softly binarize channels with \(p=\operatorname{sigmoid}(z/\tau)\), evaluate sampled quadratic forms \(q_r\) using \(q_r(x+y)=q_r(x)+q_r(y)+x^\top Jy\pmod 2\), and add the displayed \(\mathcal{L}_{\mathrm{symp}}\) term. Generate \(U\) from cheap symplectic basis operations: swap two coordinate pairs, swap members within a pair with sign adjustment, and apply elementary shears; these satisfy \(U^\top JU=J\). The paper supplies the exact cocycle and dimension count; the real-valued relaxation, gate, temperature \(\tau\), and sampled transformation distribution are engineering choices. The first experiment should use a two-layer MLP on CIFAR-10 embeddings or a molecular graph network, with \(d=16\) or \(32\), comparing a dense quadratic layer, a standard MLP, and this tied layer at equal hidden width. Measure validation accuracy, parameter count, and training loss per FLOP. A positive result is equal-or-better accuracy with at least 25% fewer interaction parameters, or faster convergence under random symplectic channel permutations.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.