# Orthogonal symmetric pair embedding

- ID: 1293
- Canonical URL: https://synthcore.org/idea/1293/orthogonal-symmetric-pair-embedding
- API JSON: https://synthcore.org/api/idea/1293.json
- API Markdown: https://synthcore.org/api/idea/1293.md
- Verification status: unverified
- Source: [arXiv:2607.22751](https://arxiv.org/abs/2607.22751)
- Category: architecture
- Solves: accuracy, stability, sample-efficiency
- ML areas: graph-nn, embedding, attention, mlp
- Math tags: orthogonal-polynomials, approximation-theory, linear-algebra, geometry
- Ratings: usefulness 5/10; difficulty 3/10; novelty 7/10

## Idea description

For every unordered pair of scalar features, construct invariant coordinates from the elementary symmetric quantities s=x+y and q=xy, then feed a truncated orthogonalized polynomial basis in (s,q) to the neural network. Estimate the basis by weighted Gram-Schmidt or Cholesky whitening under the paper's triangle weight, so polynomial channels have low redundancy and controlled scale instead of requiring an unconstrained MLP to learn both symmetry and decorrelation.

## Mathematical statement

The paper uses the unit triangle T^2={(x,y): x>=0, y>=0, x+y<=1} with symmetric weight w_{alpha,gamma,kappa}(x,y)=(xy)^alpha(1-x-y)^gamma|x-y|^{2kappa+1}, where alpha, gamma, and kappa satisfy alpha>-1, gamma>-1, kappa>-1, and alpha+kappa>-3/2. Its invariant polynomial coordinates are s=x+y and q=xy; the ordered symmetric monomials are s^n q^k, and Gram-Schmidt under the weighted inner product produces p_{n,k}^{alpha,gamma,kappa}(x,y) with leading term s^n q^k. The implementation uses the finite monomial vector m_N=[s^n q^k]_{n+2k<=N}. Define the normalized weighted density rho_w=w/Z, with Z=integral_{T^2}w, and the moment Gram matrix G=E_{rho_w}[m_N m_N^T]. Since the weighted inner product is positive definite on symmetric polynomials, G is positive definite for linearly independent monomials. If G=LL^T is its Cholesky factor, phi_N=L^{-1}m_N has E_{rho_w}[phi_N phi_N^T]=I. This whitening identity is the transferable property: invariant polynomial channels are decorrelated and normalized under the selected pair distribution.

## Key formulas

- $$\mathbf{T}^{2}=\{(x,y)\in\mathbb{R}^{2}:x\geq0,\ y\geq0,\ x+y\leq1\},\qquad w_{\alpha,\gamma,\kappa}(x,y)=(xy)^{\alpha}(1-x-y)^{\gamma}|x-y|^{2\kappa+1}.$$
- $$s=x+y,\qquad q=xy,\qquad m_N(s,q)=\bigl[s^nq^k\bigr]_{n,k\in\mathbb{N}_0,\ n+2k\leq N}.$$
- $$\langle f,g\rangle_w=\int_{\mathbf{T}^{2}}f(x,y)g(x,y)w_{\alpha,\gamma,\kappa}(x,y)\,dx\,dy,\qquad G=\mathbb{E}_{(x,y)\sim\rho_w}[m_N(s,q)m_N(s,q)^T],\qquad G=LL^T,$$
- $$\phi_N(x,y)=L^{-1}m_N(x+y,xy),\qquad \mathbb{E}_{\rho_w}[\phi_N\phi_N^T]=I,\qquad \rho_w(x,y)=\frac{w_{\alpha,\gamma,\kappa}(x,y)}{\int_{\mathbf{T}^{2}}w_{\alpha,\gamma,\kappa}(u,v)\,du\,dv}.$$

## Implementation notes

Integrate the construction at the edge or pair-embedding input of a graph neural network, set transformer, or attention score network whose pair representation must be invariant under swapping two entities. Normalize each scalar pair (a,b) into x,y in the triangle, for example by using x=sigmoid(a), y=(1-x)sigmoid(b); if the original variables are already bounded and satisfy the simplex constraint, use them directly. Choose degree N=4 or 6 and enumerate all (n,k) satisfying n+2k<=N.

Pseudocode:
1. Collect M pilot pairs (x_i,y_i) from the training distribution, or sample them from the triangle.
2. Compute r_i=(x_i y_i)^alpha(1-x_i-y_i)^gamma|x_i-y_i|^(2kappa+1), using epsilon floors for numerical singularities.
3. Form m_i=[(x_i+y_i)^n(x_i y_i)^k] and estimate G=sum_i r_i m_i m_i^T/sum_i r_i.
4. Add lambda I, with lambda=1e-6 trace(G)/d, compute the Cholesky factor G+lambda I=LL^T, and freeze L.
5. At every forward pass return phi_i=L^{-1}m_i, concatenate it with learned pair features, and apply the existing edge MLP or attention projection.

The paper supplies the invariant coordinates, admissible weight, monomial ordering, and orthogonality principle. The engineer estimates G empirically and monitors its condition number and the empirical covariance of phi. First test a two-body regression problem and a small molecular or graph benchmark against raw [x,y], a hand-symmetrized MLP on [x+y,xy], and the orthogonal embedding at equal parameter count. Success is lower validation error or faster loss descent, stable feature variance, and identical outputs after swapping each pair. Vary kappa as an ablation: positive kappa emphasizes separated pairs by downweighting the diagonal, whereas kappa near -1/2 removes the additional diagonal factor.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.