{
 "artifacts": null,
 "category": "regularization",
 "description": "Model the integer token loads of a mixture-of-experts layer as a canonical occupancy system with a fixed total number of tokens. A distributed routing phase persists while the normalized load is below a critical value; beyond that point, the excess load is either allowed to condense into a designated overflow expert or penalized if expert collapse is undesirable. The key benefit is an explicit transition criterion and finite-batch fluctuation diagnostic for routing collapse.",
 "formulas_latex": [
  "$$\\mathcal Z_N=\\sum_{\\{n_{\\mathbf k}\\}:\\,\\sum_{\\mathbf k}n_{\\mathbf k}=N}\\exp\\left[-\\beta\\sum_{\\mathbf k}n_{\\mathbf k}\\epsilon_{\\mathbf k}\\right],\\qquad \\epsilon_{\\mathbf k}=\\frac{\\hbar^2 k^2}{2m}.$$",
  "$$\\varrho_c=\\zeta(3/2),\\qquad f_0(\\varrho)=\\frac{\\langle N_0\\rangle}{N}\\simeq\\max\\left(0,1-\\frac{\\varrho_c}{\\varrho}\\right).$$",
  "$$P_\\beta(\\mathbf n)=\\frac{1}{Z_B}\\mathbf 1\\!\\left\\{\\sum_{e=1}^{M}n_e=B\\right\\}\\exp[-\\beta E_\\theta(\\mathbf n)],\\qquad Z_B=\\sum_{\\mathbf n:\\sum_e n_e=B}e^{-\\beta E_\\theta(\\mathbf n)}.$$",
  "$$f_{\\mathrm{cond}}(\\varrho)=\\max\\left(0,1-\\frac{\\varrho_c}{\\varrho}\\right),\\qquad \\sigma_{\\mathrm{cond}}(L)\\propto L^{-1}=V^{-1/3}\\quad(\\varrho\u003e\\varrho_c).$$"
 ],
 "id": 2723,
 "implementation": "1. Integration point: apply this to a top-1 or top-2 MoE router after computing logits a_te for token t and expert e. Preserve the learned router logits, but add a canonical occupancy controller that observes the complete batch load vector n=(n_1,...,n_M), with sum_e n_e=B. Define E_theta(n)=sum_e U_theta(n_e), where U_theta is a fixed or learned saturating occupancy potential, such as a quadratic capacity cost below capacity c and a softened linear overflow cost above c. 2. Pseudocode: compute router logits; route provisionally; count expert loads n_e; estimate varrho=B/M_eff and an empirical critical density varrho_hat_c from the onset of a persistent largest-expert fraction; calculate f_target=max(0,1-varrho_hat_c/varrho); add lambda times max(0,n_max/B-f_target)^2 to the router loss, or reserve an overflow expert expected to receive approximately f_target B tokens; update lambda with a slow controller. 3. Computed from the paper: the fixed-total-load canonical construction, the saturation/condensation interpretation, varrho_c=zeta(3/2) for the ideal three-dimensional reference, and the V^{-1/3} fluctuation law. Estimated empirically: effective M_eff, router-specific varrho_hat_c, potential parameters, and whether routing loads actually follow the assumed Gibbs model. 4. First cheap experiment: train a Switch-style 8-expert MLP on CIFAR-10 or WikiText-2, comparing standard load-balancing loss against this controller over batch sizes B in {128,256,512,1024,2048} and several router temperatures. Plot largest-expert fraction and routing entropy versus varrho. The falsifiable signature is a kink or rapid slope change near a reproducible varrho_hat_c; above it, the excess fraction should approximately follow 1-varrho_hat_c/varrho, while the standard deviation of the largest-expert fraction should decay approximately as B^{-1/3} rather than an unconstrained collapse or ordinary B^{-1/2} law. Reject the method if no threshold, linear excess-fraction regime, or improvement in expert utilization and validation loss appears.",
 "math_summary": "For bosonic occupancies n_k with total occupancy N, the canonical partition function is Z_N=sum_{n:sum_k n_k=N} exp[-beta sum_k n_k epsilon_k]. In three dimensions, the excited-state density saturates at varrho_c=zeta(3/2), so the thermodynamic condensate fraction is f_0=max(0,1-varrho_c/varrho) for total density varrho. The neural adaptation assigns each token to an expert e with integer load n_e, fixed total load sum_e n_e=B, and Gibbs weight P(n) proportional to exp[-beta E(n)]. Define an effective routing density varrho=B/M_eff, where M_eff is the number of capacity-normalized non-overflow expert states. The BEC transfer predicts a transition when varrho is approximately varrho_c and an excess-load fraction f_cond approximately max(0,1-varrho_c/varrho). For a linear box size L with fixed density, the paper predicts normalized condensate fluctuations sigma_0 proportional to L^{-1}=V^{-1/3} in the condensed regime; in routing, test the analogous finite-batch decay after matching the effective dimensionality.",
 "math_tags": [
  "statistical-mechanics",
  "probability",
  "large-deviations",
  "information-theory"
 ],
 "ml_areas": [
  "moe-routing",
  "training",
  "regularization"
 ],
 "paper": {
  "arxiv_id": "2608.24625",
  "arxiv_url": "https://arxiv.org/abs/2608.24625",
  "summary_what_math_gives_to_ml": "The paper provides a constructive large-deviations description of Bose-Einstein condensation in the canonical ensemble: a fixed total mass is distributed among modes, and above a critical density the excess mass concentrates into one ground-state mode while the remaining modes remain at a saturated normal density. The transferable mechanism is a sharp competition between an entropic distributed phase and a condensed phase, with a computable threshold and finite-size fluctuation law. A direct neural analogue is expert routing or memory allocation, where token assignments are constrained by a fixed batch and a controllable Gibbs energy; the BEC asymptotics predict when routing remains distributed and when one expert absorbs excess load. This yields a measurable phase diagram and a principled regularizer or capacity schedule rather than an ad hoc anti-collapse penalty.",
  "title": "Ideal Bose-Einstein condensation in the canonical ensemble: exact asymptotic estimates from large deviations",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 7,
  "usefulness": 6
 },
 "solves": [
  "stability",
  "scalability",
  "accuracy"
 ],
 "title": "BEC-Inspired Expert Condensation Control",
 "url": "https://synthcore.org/idea/2723/bec-inspired-expert-condensation-control",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}