# Branching-Fanout Sparse MoE

- ID: 815
- Canonical URL: https://synthcore.org/idea/815/branching-fanout-sparse-moe
- API JSON: https://synthcore.org/api/idea/815.json
- API Markdown: https://synthcore.org/api/idea/815.md
- Verification status: unverified
- Source: [arXiv:2607.12564](https://arxiv.org/abs/2607.12564)
- Category: architecture
- Solves: scalability, sample-efficiency, inference-speedup
- ML areas: moe-routing, mlp, inference-speedup, training-dynamics
- Math tags: probability, stochastic-processes, approximation-theory
- Ratings: usefulness 5/10; difficulty 6/10; novelty 8/10

## Idea description

Replace a fixed top-k expert count with a stochastic, token-specific fanout generated by a supercritical binary branching process stopped at a geometric time. The resulting number of active experts has finite mean but a power-law tail with log-periodic modulation, allowing most tokens to use little compute while reserving larger computation for difficult or ambiguous tokens.

## Mathematical statement

The paper studies the probability generating function $g(x)=\mathbb{E}[x^X]$ of a stopped branching population $X$ with initial population $j$. One generation maps each individual to $Y=1+\operatorname{Ber}(p)$ offspring, so the offspring pgf is $f(x)=\mathbb{E}[x^Y]=(1-p)x+px^2$. The fixed-point equation $(1+2p)g(x)=2px^j+g(f(x))$ is equivalent to $g(x)=a x^j+b g(f(x))$, where $a=2p/(1+2p)$ and $b=1/(1+2p)$. Thus sample $N\sim\operatorname{Geometric}(a)$ on $\{0,1,\ldots\}$ with $\Pr(N=n)=a b^n$, start with $j$ particles, and apply $N$ generations of the offspring rule; then $X$ is the population after the stopped process. Its mean is $\mathbb{E}[X]=2j$, because the one-generation mean multiplier is $\lambda=1+p$ and $a\sum_{n\ge0}b^n\lambda^n=2$. The coefficient asymptotic is $\Pr(X=k)=k^{-1-\rho}\Psi_j(\log_\lambda k)+o(k^{-1-\rho})$, where $\rho=\log(1+2p)/\log(1+p)$, $\lambda=1+p$, and $\Psi_j$ is positive and period-one in its argument. We use the explicit stochastic recursion, finite expected fanout, and multiplicative-scale tail; the asymptotic is a diagnostic for whether an implementation reproduces the intended allocation distribution.

## Key formulas

- $$(1+2p)\,g(x)=2p\,x^{j}+g\bigl((1-p)x+px^{2}\bigr),\qquad g(x)=\sum_{k=j}^{\infty}p_kx^k,$$
- $$g(x)=a x^j+b g(f(x)),\qquad f(x)=(1-p)x+px^2,\quad a=\frac{2p}{1+2p},\quad b=\frac{1}{1+2p},$$
- $$p_k=\Pr(X=k)=k^{-1-\rho}\,\Psi_j(\log_{\lambda}k)+o(k^{-1-\rho}),\qquad \lambda=1+p,\quad \rho=\frac{\log(1+2p)}{\log(1+p)},$$
- $$\widetilde X=\min\!\left(X,K_{\max}\right),\qquad \mathbb{E}[X]=2j,$$

## Implementation notes

Integrate this at the sparse-MoE router, after the router has produced a token difficulty score but before expert dispatch. Use a small base fanout $j$ and generate a random requested fanout independently for each token, or use the same random seed across ablations. Pseudocode: (1) input token representations $h_b$ and router scores $r_{b,e}$; (2) compute a difficulty score $d_b=\operatorname{softplus}(w^\top h_b)$; (3) choose $p_b=\sigma(c_0+c_1d_b)$, clipped to $[p_{\min},p_{\max}]$; (4) draw $N_b$ by repeatedly continuing with probability $b_b=1/(1+2p_b)$, stopping with probability $a_b=2p_b/(1+2p_b)$; (5) initialize a population of $j$ dispatch slots and for each of $N_b$ generations replace every slot by one child with probability $1-p_b$ or two children with probability $p_b$; (6) set requested fanout to $K_b=\min(X_b,K_{\max})$; (7) select the top $K_b$ experts according to $r_{b,e}$ and dispatch the token, while applying capacity overflow handling. The paper’s formula determines the stopping probabilities and offspring law. Estimate empirically the realized mean, tail exponent, and periodic residual by logging $(K_b)$; do not assume the capped mean remains exactly $2j$. Add a compute regularizer $\lambda_c\max(0,\frac1B\sum_bK_b-C)^2$ and optionally a router loss encouraging larger $K_b$ only when validation loss falls. Start with a 12-layer, 8-expert Switch-style MLP on CIFAR-100 or WikiText-2, comparing fixed top-1/top-2 routing and learned stochastic fanout at equal average dispatched experts. The first success signal is lower validation loss or perplexity at the same average expert FLOPs; secondary signals are improved performance on high-loss tokens and a predictable heavy-tailed but bounded dispatch histogram. Stress-test $K_{\max}$, router noise, and expert-capacity overflow, since the uncapped mathematical tail can create severe batch imbalance.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.