Lie-Exchange Configuration Mixer
Implementation & benchmark of arXiv:2607.13208 — The Action of the Lie Algebra $\mathfrak{sl}_n$ on Colored Graphs and Multicolored Johnson Graphs
Source paper: The Action of the Lie Algebra $\mathfrak{sl}_n$ on Colored Graphs and Multicolored Johnson Graphs arXiv:2607.13208 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Add a profile-preserving exchange layer that mixes representations through pairwise state swaps, with weights shared over all positions. Unlike ordinary token-wise MLP mixing, the layer explicitly models interactions between two categorical states while respecting the symmetry of position permutations.
Formulas
Mathematical statement
For $m$ positions with $n$ possible states, define $E_{ab}^{(p)}$ as the matrix unit $E_{ab}$ acting on position $p$. The global root operators of the paper's Lie-algebra action are $F_{ab}=\sum_{p=1}^mE_{ab}^{(p)}$ for $a\neq b$; they convert one state-$b$ position into state $a$. A profile-preserving exchange must apply a lowering operation at one position and the reverse raising operation at another, giving $A_\alpha=\sum_{p<q}\sum_{a\neq b}E_{ab}^{(p)}E_{ba}^{(q)}$. For neural features $h_p\in\mathbb{R}^d$ and state probabilities $\pi_{p,a}=\operatorname{softmax}(W h_p)_a$, use the differentiable expected exchange message $M_p=\sum_{q\neq p}\sum_{a\neq b}\pi_{p,b}\pi_{q,a}\,V_{ab}h_q$. A normalized residual layer uses $\widetilde M_p=M_p/\max(1,\sum_{q\neq p}\sum_{a\neq b}\pi_{p,b}\pi_{q,a})$ and $h'_p=h_p+\gamma\,\sigma(\widetilde M_p)$, where $\gamma$ is learned or initialized small.
Implementation notes
Insert the layer after edge-token embeddings and before a graph Transformer or pooled graph head. The input is h[B,m,d]; compute state logits z=W_state(h) and probabilities pi=softmax(z, dim=-1). For each ordered state pair $(a,b)$ with $a\neq b$, form src_b = pi[:,:,b] * h and aggregate it across positions, for example with global_a = sum_q pi[:,:,a,q] * h_q; apply a learned matrix V[a,b] and broadcast the result back with coefficient pi[:,:,b]. An efficient implementation is M[p] = sum_{a != b} pi[p,b] * V[a,b] @ sum_{q != p}(pi[q,a] * h[q]); subtract the self-term explicitly so that $q=p$ is excluded. Normalize by the displayed denominator, apply GELU or SiLU, and use a residual coefficient gamma initialized to $10^{-2}$. For large $m$, use sampled position pairs or low-rank factorization V[a,b]=U_a R V_b^T, reducing the naive $O(m^2n^2d)$ cost. The exact mathematical object is the unequal-state exchange operator; the soft probabilities and feature matrices are the neural relaxation. Start with a 4-layer edge-token model on fixed-profile synthetic colored graphs and compare against a shared MLP, a standard self-attention block with matched parameter count, and a token-wise state mixer. Check permutation invariance by randomly relabeling edge positions, profile preservation of the state head, validation accuracy at equal FLOPs, and optimization stability. Success is improved accuracy or faster loss descent, especially when labels depend on relations between state counts rather than individual edges.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.