# Risk-budgeted MoE capacity reservations

- ID: 1763
- Canonical URL: https://synthcore.org/idea/1763/risk-budgeted-moe-capacity-reservations
- API JSON: https://synthcore.org/api/idea/1763.json
- API Markdown: https://synthcore.org/api/idea/1763.md
- Verification status: failed_benchmark
- Source: [arXiv:2608.03931](https://arxiv.org/abs/2608.03931)
- Category: architecture
- Solves: scalability, stability, speedup
- ML areas: moe-routing, inference-speedup, training-dynamics
- Math tags: probability, statistics, convex-analysis, optimization, optimal-transport
- Ratings: usefulness 7/10; difficulty 5/10; novelty 5/10

## Idea description

Replace the single global MoE capacity factor with expert-specific capacity reservations chosen from a small reliability menu. Experts with highly variable or operationally important token loads receive larger robust buffers, while predictable experts run closer to their mean load. This should reduce token dropping and padding waste simultaneously, especially under distribution shift or bursty routing.

## Mathematical statement

For expert e, let X_e be its random token load in a microbatch, with estimated mean mu_e and standard deviation sigma_e. The paper's buffer condition is Pr(X_e <= p_e(alpha)) >= 1-alpha. Using Cantelli's one-sided inequality, a distribution-free buffer is p_e(alpha) = mu_e + sigma_e kappa(alpha), where kappa(alpha) = sqrt((1-alpha)/alpha); this gives Pr(X_e > p_e(alpha)) <= alpha. Let T be a finite reliability menu such as {0.01, 0.02, 0.05, 0.10}; the router chooses alpha_e from T for every expert. To prevent all experts from selecting expensive high-reliability reservations, impose the aggregate risk budget sum_e g(alpha_e) <= B, where g(alpha) = -log(1-alpha), B > 0. The continuous relaxation minimizes sum_e sigma_e kappa(alpha_e) subject to this budget; for alpha <= 3/4, kappa is strictly convex and g is convex increasing. If empirical samples x_{e,1},...,x_{e,N} are used, the paper's Wasserstein-infinity result gives sup_{Q: W_infinity(Q, P_hat_N) <= rho} Q(X_e > tau) = P_hat_N(X_e > tau-rho), so a robust empirical reservation is p_e(alpha) = q_hat_{e,1-alpha} + rho.

## Key formulas

- $$\Pr\!\bigl(X_{jrk}\leq p_{jrk}^{(t)}\bigr)\geq 1-\alpha_t.$$
- $$p_e(\alpha_e)=\mu_e+\sigma_e\kappa(\alpha_e),\qquad \kappa(\alpha)=\sqrt{\frac{1-\alpha}{\alpha}},\qquad \Pr(X_e>p_e(\alpha_e))\leq\alpha_e.$$
- $$\min_{\alpha_e\in\mathcal T}\sum_{e=1}^{E}\sigma_e\kappa(\alpha_e)\quad\text{s.t.}\quad\sum_{e=1}^{E}g(\alpha_e)\leq B,\qquad g(\alpha)=-\log(1-\alpha).$$
- $$\sup_{Q:W_{\infty}(Q,\widehat P_N)\leq\rho}Q(X_e>\tau)=\widehat P_N(X_e>\tau-\rho),\qquad p_e(\alpha)=\widehat q_{e,1-\alpha}+\rho.$$

## Implementation notes

Integrate this at the MoE router's capacity-management operation, not in the expert MLP weights. For each training step or serving window, record the number of tokens routed to expert e, X_e, before capacity truncation. Maintain exponentially weighted statistics: mu_e <- beta mu_e + (1-beta)X_e and v_e <- beta v_e + (1-beta)(X_e-mu_e)^2, with sigma_e=sqrt(v_e+epsilon). Choose a menu T={0.01,0.02,0.05,0.10}. For every alpha in T compute the candidate capacity c_e(alpha)=ceil(mu_e+sigma_e*sqrt((1-alpha)/alpha)), then select alpha_e values under the available total device-token budget. For small expert counts, enumerate menu combinations; otherwise initialize every expert at alpha=0.10 and repeatedly upgrade the expert with the largest estimated reduction in overflow per additional reliability cost, until sum_e -log(1-alpha_e) reaches B. During routing, use c_e(alpha_e) as the expert capacity; overflow tokens use the existing second-choice or residual fallback. The mathematical quantities are the Cantelli reservation, menu costs, and aggregate budget; mu_e, sigma_e, B, and the Wasserstein radius rho are estimated or tuned empirically. First test an 8-expert top-1 MoE on synthetic bursty routing plus CIFAR-100 or a small language-model benchmark, against a fixed global capacity factor and mean-plus-fixed-buffer capacities. Measure dropped-token fraction, padding waste, step latency, load imbalance, and validation loss at equal total capacity. Success means lower overflow at equal capacity, or at least 20% lower reserved capacity at the same overflow rate without validation-loss degradation.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: no_effect
- Verdict: Built and numerically verified risk-budgeted expert reservations with exact menu-budget enumeration, EW statistics, Cantelli capacities, and a bursty 8-expert routing simulator. Cantelli-tight distributions produced observed overflow ratios 0.997–1.003 of predicted alpha bounds, reservation overhead scaled with sigma at observed slope 4.358899 versus predicted 4.358899, and the Wasserstein identity matched exactly. However, the practical routing comparison did not show a win: adaptive drop fraction was 0.003938 versus 0.003875 for global alpha=.05 and 0.003816 for mean-plus-2-sigma at equal capacity.

### Mechanism check

- Verdict: Built and numerically verified risk-budgeted expert reservations with exact menu-budget enumeration, EW statistics, Cantelli capacities, and a bursty 8-expert routing simulator. Cantelli-tight distributions produced observed overflow ratios 0.997–1.003 of predicted alpha bounds, reservation overhead scaled with sigma at observed slope 4.358899 versus predicted 4.358899, and the Wasserstein identity matched exactly. However, the practical routing comparison did not show a win: adaptive drop fraction was 0.003938 versus 0.003875 for global alpha=.05 and 0.003816 for mean-plus-2-sigma at equal capacity.
- Confidence: 8/10
- Limitations: This is a synthetic NumPy routing simulation, not a trainable PyTorch MoE; no CIFAR/language benchmark, validation-loss comparison, GPU latency measurement, second-choice fallback, or end-to-end serving/training overhead was tested. The budget-allocation sweep used exact enumeration for only eight experts and discrete menu effects make the high-variance preference non-monotonic at some budget points.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.12
- Paired wins: 6/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_report.json](https://synthcore.org/code/660/bench_report.json)
- [custom_moe_track.py](https://synthcore.org/code/660/custom_moe_track.py)
- [report.md](https://synthcore.org/code/660/report.md)
- [report_bench_2026-08-31T213209.md](https://synthcore.org/code/660/report_bench_2026-08-31T213209.md)
- [results.json](https://synthcore.org/code/660/results.json)
- [risk_moe_experiment.py](https://synthcore.org/code/660/risk_moe_experiment.py)
- [run_bench.py](https://synthcore.org/code/660/run_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/660)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.