# Grassmannian Tropical Router

- ID: 1171
- Canonical URL: https://synthcore.org/idea/1171/grassmannian-tropical-router
- API JSON: https://synthcore.org/api/idea/1171.json
- API Markdown: https://synthcore.org/api/idea/1171.md
- Verification status: unverified
- Source: [arXiv:2607.20109](https://arxiv.org/abs/2607.20109)
- Category: architecture
- Solves: stability, scalability, accuracy
- ML areas: moe-routing, transformer, architecture
- Math tags: combinatorics, geometry, algebra, optimization
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Replace unconstrained MoE router logits with structured phase scores indexed by N-subsets of M ordered parameters. Each token is assigned to the dominant phase, while neighboring routing regions obey the Grassmannian rule that adjacent labels share N-1 indices, reducing arbitrary fragmented decision boundaries and encouraging smooth expert transitions.

## Mathematical statement

The paper uses phase functions indexed by K in the set of N-element subsets of [M]. For angular parameters psi_j, the phase is Theta_K(x,y,t) = 2x sum_{k in K} sin(psi_k) + 2y sum_{k in K} cos(psi_k) - 2t sum_{k in K} sin(2 psi_k). Here x,y,t are three coordinates, M is the number of base parameters, N is the subset size, and psi_k are generic ordered angles. The dominant region is argmax_K Theta_K(x,y,t) plus an optional log Plucker weight log Delta_K(A), where Delta_K(A) is the K-th N-by-N minor of a totally nonnegative matrix A. The paper proves that, under genericity, adjacent dominant regions have labels sharing N-1 indices. In the asymptotic Toda construction, a boundary associated with indices a,b has equation x_1 + n(ln(lambda_a)-ln(lambda_b))/(lambda_a-lambda_b) - x_{-1}/(lambda_a lambda_b) = 0, and its slope is (lambda_b-lambda_a)/(ln(lambda_a)-ln(lambda_b)). We exploit the explicit additive subset structure and adjacency property as a structured router prior.

## Key formulas

- $$\Theta_K(x,y,t)=2x\sum_{k\in K}\sin\psi_k+2y\sum_{k\in K}\cos\psi_k-2t\sum_{k\in K}\sin 2\psi_k,$$
- $$s_K(z)=\Theta_K(x(z),y(z),t(z))+\log w_K,\qquad r(z)=\operatorname*{argmax}_{K\in\binom{[M]}{N}}s_K(z),$$
- $$L_{[a,b]}^{(n)}:x_1+n\frac{\phi_a-\phi_b}{\lambda_a-\lambda_b}-\frac{1}{\lambda_a\lambda_b}x_{-1}=0,\qquad \phi_i=\ln\lambda_i,$$
- $$\operatorname{softmax}_K(s/\tau)=\frac{\exp(s_K/\tau)}{\sum_{J\in\binom{[M]}{N}}\exp(s_J/\tau)}.$$

## Implementation notes

Integrate this module at the router-logit stage of a small top-1 or top-2 mixture-of-experts Transformer. Given token representation h in R^d, use three learned scalar projections x(h)=u_x^T h, y(h)=u_y^T h, and t(h)=u_t^T h, optionally normalized with LayerNorm. Choose M ordered trainable parameters psi_1<...<psi_M by parameterizing psi_j=psi_min+softplus(cumsum_j(delta)) and clipping them to an interval shorter than pi/2 to avoid degeneracies. Enumerate all N-subsets K of {1,...,M}. Precompute c_K=sum_{k in K} sin(psi_k), d_K=sum_{k in K} cos(psi_k), and e_K=sum_{k in K} sin(2 psi_k), then compute s_K=2*x*c_K+2*y*d_K-2*t*e_K+b_K, where b_K is a learned log routing weight. During training, use softmax(s/tau) or noisy softmax for load balancing; anneal tau from 1.0 to 0.1, and use argmax/top-k at inference. For a strict Grassmannian prior, initialize b_K=0 and use a Vandermonde positive matrix A_{rj}=lambda_j^(r-1), with 0<lambda_1<...<lambda_M, then set b_K=alpha*log(det(A[:,K])+epsilon), where det(A[:,K]) is the Plucker minor. The mathematical quantities are the subset phase coefficients and boundary geometry; empirical quantities are the projections, psi values, and expert losses. Add an auxiliary loss penalizing router load imbalance and, optionally, a locality penalty that samples token pairs near a predicted phase boundary and encourages only subsets differing by one index to compete. First test on a 4-expert, 2-layer Transformer trained on WikiText-103 or CIFAR-10 sequence modeling, comparing a standard linear top-1 router at matched parameters and FLOPs. Measure validation perplexity or accuracy, expert-load entropy, number of disconnected routing components, and loss variance. Success means equal-or-better validation quality with lower routing collapse and fewer expert switches for neighboring token embeddings; a stronger result is improved quality at the same active-expert FLOPs.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.