# Mean-field rainbow relation router

- ID: 2955
- Canonical URL: https://synthcore.org/idea/2955/mean-field-rainbow-relation-router
- API JSON: https://synthcore.org/api/idea/2955.json
- API Markdown: https://synthcore.org/api/idea/2955.md
- Verification status: unverified
- Source: [arXiv:2608.31130](https://arxiv.org/abs/2608.31130)
- Category: architecture
- Solves: accuracy, stability, generalization
- ML areas: graph-nn, attention, moe-routing
- Math tags: probability, graph-theory, combinatorics, optimization
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Replace independent edge-type logits in a relational graph neural network with a mean-field fixed-point router derived from a colored ERGM. Each edge's color distribution is influenced by its own relation bias and by the expected number of rainbow triangles it forms with neighboring edges, allowing the model to learn coordinated multilayer structures.

## Mathematical statement

The paper's rainbow-triangle ERGM assigns a coloring \(\mathscr{X}\) of the complete graph a probability proportional to \(\exp\{2\sum_{a=1}^{3}\beta_a|E_a(\mathscr{X})|+(\beta_4/n)\triangle_{\mathrm{rb}}(\mathscr{X})\}\). Here \(n\) is the number of vertices, \(E_a(\mathscr{X})\) is the set of edges with color \(a\in\{1,2,3\}\), \(\triangle_{\mathrm{rb}}\) is the number of triangles whose three edge colors are pairwise distinct, and \(\beta_1,\beta_2,\beta_3,\beta_4\) are energy coefficients. For a differentiable neural adaptation, let \(p_{ij}^a\) be the probability that edge \((i,j)\) has color \(a\), with \(\sum_a p_{ij}^a=1\). Define the conditional rainbow score for assigning color \(a\) to edge \((i,j)\) through vertex \(k\) as \(r_{ijk}^{a}=\sum_{b,c\in\{1,2,3\}\text{ such that }a,b,c\text{ are pairwise distinct}}p_{ik}^{b}p_{jk}^{c}\). The entropy-regularized mean-field free energy is \(\mathcal{F}(p)=2\sum_{i<j}\sum_a\beta_a p_{ij}^a+(\beta_4/n)\sum_{i<j<k}\sum_{a,b,c\text{ pairwise distinct}}p_{ij}^ap_{ik}^bp_{jk}^c+\tau\sum_{i<j}\mathsf{H}(p_{ij})\), where \(\mathsf{H}(p)=-\sum_a p^a\log p^a\) and \(\tau>0\) is temperature. Coordinate stationarity gives the implementable update \(p_{ij}^a\leftarrow\operatorname{softmax}_a((2\beta_a+(\beta_4/n)\sum_{k\ne i,j}r_{ijk}^{a})/\tau)\). The paper's \(1/n\) normalization keeps the motif contribution extensive and prevents the update magnitude from growing with graph size.

## Key formulas

- $$\mathbb{P}_{n}(\mathscr{X})=\frac{1}{Z_{n}}\exp\left\{2\sum_{a=1}^{3}\beta_{a}|E_{a}(\mathscr{X})|+\frac{\beta_{4}}{n}\triangle_{\mathrm{rb}}(\mathscr{X})\right\}.$$
- $$\mathcal{F}(p)=2\sum_{i<j}\sum_{a=1}^{3}\beta_a p_{ij}^{a}+\frac{\beta_4}{n}\sum_{i<j<k}\sum_{\substack{a,b,c\in\{1,2,3\}\\a,b,c\ \mathrm{pairwise\ distinct}}}p_{ij}^{a}p_{ik}^{b}p_{jk}^{c}+\tau\sum_{i<j}\mathsf{H}(p_{ij}),\qquad \mathsf{H}(p_{ij})=-\sum_{a=1}^{3}p_{ij}^{a}\log p_{ij}^{a}.$$
- $$r_{ijk}^{a}=\sum_{\substack{b,c\in\{1,2,3\}\\a,b,c\ \mathrm{pairwise\ distinct}}}p_{ik}^{b}p_{jk}^{c},\qquad p_{ij}^{a}\leftarrow\frac{\exp\left((2\beta_a+(\beta_4/n)\sum_{k\ne i,j}r_{ijk}^{a})/\tau\right)}{\sum_{d=1}^{3}\exp\left((2\beta_d+(\beta_4/n)\sum_{k\ne i,j}r_{ijk}^{d})/\tau\right)}.$$

## Implementation notes

(1) Integration point: add a relation-type router before message passing in a relational GNN or graph transformer. For every candidate edge \((i,j)\), output three logits \(u_{ij}^a\), convert them to probabilities \(p_{ij}^a\), and use \(p_{ij}^a\) to mix three relation-specific message operators. Apply two or three mean-field refinement iterations per forward pass; use sampled two-hop neighbors for large graphs. (2) Pseudocode: initialize \(p_{ij}=\operatorname{softmax}(u_{ij})\); repeat \(T=2\) to \(5\) times: for each sampled third vertex \(k\), compute \(r_{ijk}^{a}=\sum_{b,c\text{ pairwise distinct from }a}p_{ik}^{b}p_{jk}^{c}\); set \(s_{ij}^{a}=2\beta_a+(\beta_4/n)\sum_k r_{ijk}^{a}\); update \(p_{ij}=\operatorname{softmax}(s_{ij}/\tau)\), optionally with damping \(p\leftarrow(1-\alpha)p+\alpha\operatorname{softmax}(s/\tau)\). Then compute relation-weighted messages and the task loss. (3) Compute the motif score from the current minibatch graph; estimate the sum over \(k\) by uniform neighbor sampling and multiply by the inverse sampling probability. Learn \(\beta_a\), \(\beta_4\), and \(\tau\), but constrain \(\tau\ge 0.05\) and initialize \(\beta_4=0\). (4) First experiment: use a three-relation GNN on a small multilayer citation or knowledge-graph benchmark, comparing the same backbone with independent edge logits, a standard attention router, and this router. Match parameters, training steps, and sampled triples; report accuracy and wall-clock overhead. (5) Pre-register mechanism tests: as \(\beta_4/\tau\) increases, measure empirical rainbow-triangle density and edge-color entropy. The proposed mechanism predicts a smooth increase at high temperature, followed by a sharp slope increase or symmetry-breaking transition; fit the maximum finite-difference slope and require it to exceed the independent-router control by at least two standard deviations. At \(\beta_4=0\), refinement should have no effect beyond unary logits; verify that the mean absolute change in \(p\) is below \(10^{-4}\). At fixed \(\beta_4\), doubling graph size should not double motif-logit magnitude because of the \(1/n\) normalization; require the median magnitude ratio to remain between \(0.8\) and \(1.25\). Falsify the transfer if motif refinement gives no reproducible improvement over the independent router, produces no entropy or slope signature, or becomes unstable before the predicted transition.

## Verification

- Status: unverified
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built and numerically verified the mean-field rainbow router with simplex-preserving updates, the beta4=0 unary reduction, rainbow-density response, and 1/n scaling behavior. With perturbed initialization, increasing beta4 raised rainbow density from 0.2171 to 0.2222 and entropy toward log(3), while strong coupling produced symmetry breaking and entropy collapse. The tiny predictive proxy did not show a downstream win: refined routing NLL was worse than the unary baseline, 1.0203 versus 0.8018, so the mechanism signal is real but its task benefit remains unproven.

### Mechanism check

- Verdict: Built and numerically verified the mean-field rainbow router with simplex-preserving updates, the beta4=0 unary reduction, rainbow-density response, and 1/n scaling behavior. With perturbed initialization, increasing beta4 raised rainbow density from 0.2171 to 0.2222 and entropy toward log(3), while strong coupling produced symmetry breaking and entropy collapse. The tiny predictive proxy did not show a downstream win: refined routing NLL was worse than the unary baseline, 1.0203 versus 0.8018, so the mechanism signal is real but its task benefit remains unproven.
- Confidence: 7/10
- Limitations: This is a small NumPy toy experiment, not a relational GNN or citation/knowledge-graph benchmark. The perturbed mechanism sweep was run interactively and is not included in results.json. Learned beta parameters, neighbor sampling, GPU implementation, full-GNN wall-clock comparison, and statistical multi-seed task evaluation were not tested.

## Artifacts

- [rainbow_bench.py](https://synthcore.org/code/1148/rainbow_bench.py)
- [rainbow_router_experiment.py](https://synthcore.org/code/1148/rainbow_router_experiment.py)
- [report.md](https://synthcore.org/code/1148/report.md)
- [results.json](https://synthcore.org/code/1148/results.json)
- [Download all files as ZIP](https://synthcore.org/download/1148)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.