# Exact Multi-Population Hyperplane Router

- ID: 365
- Canonical URL: https://synthcore.org/idea/365/exact-multi-population-hyperplane-router
- API JSON: https://synthcore.org/api/idea/365.json
- API Markdown: https://synthcore.org/api/idea/365.md
- Verification status: mechanism_failed
- Source: [arXiv:2607.02400](https://arxiv.org/abs/2607.02400)
- Category: architecture
- Solves: scalability, stability, accuracy
- ML areas: moe-routing, embedding, transformer
- Math tags: geometry, convex-analysis, algebra, numerical-analysis
- Ratings: usefulness 7/10; difficulty 6/10; novelty 7/10

## Idea description

Replace a learned binary MoE gate with a hyperplane whose two sides contain prescribed proportions of several token populations simultaneously. In a low-dimensional routing projection, solve the cap-volume equations directly, producing deterministic per-population load control without an auxiliary load-balancing loss. Recursively applying the construction yields a balanced binary expert tree.

## Mathematical statement

For an oriented hyperplane H(u=t), let u\in\mathbb S^{d-1} be a unit normal and define the negative halfspace P(u\le t)=\{x\in P:u^{\mathsf T}x\le t\}. For a convex polytopal measure P, its cap-volume function is C_P(u,t)=\operatorname{vol}(P\cap\{x:u^{\mathsf T}x\le t\}). On each cell of the arrangement determined by which polytope edges or faces are intersected, C_P(u,t) is a fixed rational function of (u,t). Given d polytopal measures P_1,\ldots,P_d and target fractions \alpha_i\in(0,1), the prescribed-proportion cut solves C_{P_i}(u,t)=\alpha_i\operatorname{vol}(P_i) for i=1,\ldots,d, together with u^{\mathsf T}u=1 and the inequalities defining the active cell. In the ML adaptation, P_i approximates the projected token distribution for population i, and routing is r(z)=\mathbf 1[u^{\mathsf T}z>t].

## Key formulas

- $$P(u\le t)=\{x\in P:u^{\mathsf T}x\le t\},\qquad C_P(u,t)=\operatorname{vol}(P(u\le t)).$$
- $$C_{P_i}(u,t)=\alpha_i\operatorname{vol}(P_i),\quad i=1,\ldots,d,\qquad u^{\mathsf T}u=1.$$
- $$r(z)=\mathbf 1[u^{\mathsf T}z>t],\qquad \Pr_{z\sim P_i}[r(z)=0]=\alpha_i.$$
- $$t+u_1+u_2\geq0,\quad t-2u_1+u_2\leq0,\quad t+u_1-2u_2\leq0.$$

## Implementation notes

Integrate this at the MoE router immediately before dispatching tokens to two experts. Learn or freeze a projection W\in\mathbb R^{d    imes D}, with d=2 or 3, and compute z=W h for hidden state h. For every population i, maintain a reservoir of 32--128 projected tokens and form a convex hull P_i; use an axis-aligned box as the first MVP. Set target fractions alpha_i, such as 0.5 for balanced routing. Enumerate candidate arrangement cells induced by hull edges or faces. For each cell, clip P_i by u^T x\le t, evaluate the resulting polygon or polyhedron volume, and solve C_{P_i}(u,t)=alpha_i vol(P_i) for all i plus u^T u=1. Reject roots violating the cell inequalities. If no exact root exists, minimize the normalized residual sum_i(C_{P_i}(u,t)/vol(P_i)-alpha_i)^2 and select the best candidate. Pseudocode: build P_i; generate cells; solve equations; filter feasible roots; dispatch using r(h)=1[ u^T W h>t]. The paper supplies the cap equations and semialgebraic cell constraints; hull maintenance and root solving are numerical approximations. First test on CIFAR-10 or TinyStories with a two-expert MLP, comparing Top-1 gating plus load loss against this router at equal active FLOPs. Record routing fractions, overflow rate, validation loss, and routing overhead. Success means below 1% population load imbalance without an auxiliary loss and equal-or-lower validation loss.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built an exact two-dimensional box-cap router using polygon clipping and nonlinear solving, plus a reproducible two-expert synthetic regression benchmark. The cap equations matched the prescribed 0.5 fractions with zero solver residual and at most 0.00085 Monte Carlo error; routing was balanced across both populations without a load loss. However, validation MSE was 0.1344 versus 0.0737 for the learned baseline, so the promised accuracy win was not observed despite the clear load-control effect.

### Mechanism check

- Verdict: Built an exact two-dimensional box-cap router using polygon clipping and nonlinear solving, plus a reproducible two-expert synthetic regression benchmark. The cap equations matched the prescribed 0.5 fractions with zero solver residual and at most 0.00085 Monte Carlo error; routing was balanced across both populations without a load loss. However, validation MSE was 0.1344 versus 0.0737 for the learned baseline, so the promised accuracy win was not observed despite the clear load-control effect.
- Confidence: 9/10
- Limitations: Only a synthetic two-dimensional axis-aligned-box population model and linear two-expert regressors were tested; no CIFAR-10/TinyStories, learned projection, convex-hull reservoirs, recursive expert tree, overflow capacity test, wall-clock overhead, or equal-FLOP neural MoE comparison was evaluated.

## Artifacts

- [report.md](https://synthcore.org/code/73/report.md)
- [results.json](https://synthcore.org/code/73/results.json)
- [router_experiment.py](https://synthcore.org/code/73/router_experiment.py)
- [Download all files as ZIP](https://synthcore.org/download/73)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.