# Yang–Baxter Pairwise Router

- ID: 1132
- Canonical URL: https://synthcore.org/idea/1132/yang-baxter-pairwise-router
- API JSON: https://synthcore.org/api/idea/1132.json
- API Markdown: https://synthcore.org/api/idea/1132.md
- Verification status: unverified
- Source: [arXiv:2607.18959](https://arxiv.org/abs/2607.18959)
- Category: architecture
- Solves: stability, scalability, speedup
- ML areas: moe-routing, transformer, training-dynamics
- Math tags: algebra, linear-algebra, combinatorics
- Ratings: usefulness 5/10; difficulty 6/10; novelty 8/10

## Idea description

Replace unconstrained pairwise token-routing interactions with a structured two-token router derived from an involutive set-theoretical Yang–Baxter solution. The pair operator is a convex interpolation between identity and a permutation of discrete routing states, so it cannot amplify probability mass or logits when applied to routing distributions. The Yang–Baxter relation provides a falsifiable test for whether three-token routing updates are insensitive to the two admissible overlapping-update orderings.

## Mathematical statement

The paper starts with a finite set of species X and an operator \(\check r:V\otimes V\to V\otimes V\), where V has basis \(\{|i\rangle:i\in X\}\), acting as \(\check r|i,j\rangle=|g_i(j),f_j(i)\rangle\). The maps \(f_j,g_i:X\to X\) are bijections, \(\check r\) satisfies the braid/Yang–Baxter equation \(\check r_{12}\check r_{23}\check r_{12}=\check r_{23}\check r_{12}\check r_{23}\), and it is involutive: \(\check r^2=\mathrm{Id}\). Baxterisation gives \(\check r(u)=\frac{u\check r+\mathrm{Id}_{V\otimes V}}{u+1}\), with \(u\ge0\). If \(\check r\) is represented by a permutation matrix P, then \(R(u)=(uP+I)/(u+1)=(1-a)I+aP\), where \(a=u/(u+1)\in[0,1)\). Thus R(u) is a convex combination of identity and a permutation, preserves nonnegative probability vectors and total mass, and has no amplification in \(\ell_1\). The parameter-dependent consistency relation is \(\check r_{12}(u)\check r_{23}(u+v)\check r_{12}(v)=\check r_{23}(v)\check r_{12}(u+v)\check r_{23}(u)\).

## Key formulas

- $$\check{r}\lvert i,j\rangle=\lvert g_{i}(j),f_{j}(i)\rangle,\qquad \check r_{12}\check r_{23}\check r_{12}=\check r_{23}\check r_{12}\check r_{23},\qquad \check r^2=\mathrm{Id}.$$
- $$\check{r}(u)=\frac{u\,\check{r}+\mathrm{Id}_{V\otimes V}}{u+1}=(1-a)I+aP,\qquad a=\frac{u}{u+1}.$$
- $$\check{r}_{12}(u)\,\check{r}_{23}(u+v)\,\check{r}_{12}(v)=\check{r}_{23}(v)\,\check{r}_{12}(u+v)\,\check{r}_{23}(u).$$
- $$q'_{ab}=\sum_{c,d\in X}R(u)_{ab,cd}\,q_{cd},\qquad q'=(1-a)q+aPq.$$

## Implementation notes

Use this as an experimental replacement for a small part of MoE routing rather than as a full Transformer replacement. Let each token have a categorical expert distribution \(p_t\in\Delta^{N-1}\), and form pair distributions \(q_{t,t+1}=p_t\otimes p_{t+1}\in\mathbb R^{N^2}\). Choose a concrete involutive set-theoretical solution by defining explicit bijections \(f_j,g_i\) on \(X=\{1,\ldots,N\}\), enumerate the map \((i,j)\mapsto(g_i(j),f_j(i))\), and store its permutation matrix \(P\in\mathbb R^{N^2\times N^2}\). For each adjacent pair, compute \(u_t=\mathrm{softplus}(s_t)\), set \(a_t=u_t/(1+u_t)\), and update \(q'_{t,t+1}=((u_tP+I)/(u_t+1))q_{t,t+1}\). Marginalize \(q'\) over the second and first expert indices to obtain revised distributions \(\tilde p_t,\tilde p_{t+1}\), then apply \(p_t\leftarrow(1-\rho)p_t+\rho\tilde p_t\) with \(\rho\le1\). During training, use straight-through top-1 or top-2 routing after this operation. A cheaper implementation avoids materializing P: reshape the pair tensor to \([N,N]\), apply the species permutation by indexing, and mix it with the original tensor. The mathematical quantities are exact; \(u_t,s_t,\rho\) are learned or scheduled empirically. First test on a 4-layer, 8-expert Switch-style MLP on CIFAR-100 or a small language-model corpus, comparing standard router logits against this pair router at equal FLOPs. Measure auxiliary load-balancing loss, router entropy, gradient norm spikes, dropped-token rate, and validation loss. Success means fewer routing collapses or dropped tokens and smoother loss at equal perplexity or accuracy.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.