{
 "artifacts": null,
 "category": "dynamics",
 "description": "Replace a static top-k MoE capacity rule with a router whose expert allocation evolves through a finite-domain coverage process. Experts with larger current occupancy can either receive more future capacity, intentionally amplifying specialization, or receive less capacity by reversing the size dependence, allowing a controlled test of the paper's asymmetry-amplification mechanism.",
 "formulas_latex": [
  "$$\\tau_{\\alpha}(\\mathbf r;\\mathcal D)=\\int_{0}^{\\infty}\\exp[-N_{\\alpha}(\\mathbf r,t;\\mathcal D)]\\,dt$$",
  "$$N_i(t;s_i)=\\lambda_i(s_i)t^p,\\qquad \\lambda_i(s_i)=\\lambda_0(s_i+\\epsilon)^\\beta$$",
  "$$\\tau_i(s_i)=\\int_0^\\infty e^{-\\lambda_i(s_i)t^p}\\,dt=\\frac{\\Gamma(1/p)}{p\\,\\lambda_i(s_i)^{1/p}},\\qquad v_i(s_i)=\\tau_i(s_i)^{-1}=\\frac{p}{\\Gamma(1/p)}\\lambda_i(s_i)^{1/p}$$",
  "$$s_i^{(u+1)}=\\Pi_{[0,1]}\\!\\left(s_i^{(u)}+\\eta\\left[v_i(s_i^{(u)})-\\frac1E\\sum_{j=1}^{E}v_j(s_j^{(u)})\\right]\\right),\\qquad \\dot\\delta_i\\approx v'(\\bar s)\\delta_i$$"
 ],
 "id": 2963,
 "implementation": "Integrate the mechanism into the capacity/allocation state of a top-1 or top-2 MoE router, not into the expert MLP weights. Let the router produce logits \\(z_{b,i}\\) for token \\(b\\) and expert \\(i\\), and let \\(q_{b,i}=\\operatorname{softmax}(z_b)_i\\). At each training step, compute soft occupancy \\(s_i=\\operatorname{mean}_b q_{b,i}\\), detach \\(s_i\\) when updating the allocation state, and maintain a scalar capacity multiplier \\(c_i\\). Initialize \\(c_i=1\\). For every batch compute \\(q\\), \\(s_i\\), \\(\\lambda_i=\\lambda_0(s_i+\\epsilon)^\\beta\\), and \\(v_i=p\\lambda_i^{1/p}/\\Gamma(1/p)\\). Update \\(c_i\\leftarrow\\operatorname{clip}(c_i+\\eta_c(v_i-\\operatorname{mean}_jv_j),c_{\\min},c_{\\max})\\), then renormalize so \\(\\operatorname{mean}_i c_i=1\\), and add \\(\\log c_i\\) to expert \\(i\\)'s router logit before top-k selection. Apply token dropping only after the modified logits. The paper-derived quantities are the survival integral, power-law hazard, inverse survival time, and centered velocity update; \\(\\lambda_0,p,\\beta,\\eta_c,\\epsilon\\) are engineering hyperparameters. First run a 125M-parameter Switch-style Transformer on C4 or Wikitext-103 with 8 experts, comparing static capacity, standard load-balancing loss, \\(\\beta=+1\\) amplification, and \\(\\beta=-1\\) balancing at equal tokens, optimizer steps, expert FLOPs, and total parameter count. Measure validation perplexity, dropped-token fraction, expert utilization entropy, and the growth of an intentionally injected initial occupancy perturbation. Pre-register three mechanism tests: fit \\(\\log|\\delta_u|\\) versus update number and compare its slope with \\(\\eta_c v'(1/E)\\) within 20%; verify that changing \\(\\beta\\) from \\(-1\\) to \\(+1\\) changes the slope from negative to positive with a transition near \\(\\beta=0\\); and verify that increasing \\(\\eta_c\\) increases the slope approximately linearly until clipping. Remove only the survival-time capacity update while retaining the same router and load-balancing loss. The proposal is falsified if occupancy perturbations do not show the predicted sign change, measured slopes are unrelated to \\(v'(1/E)\\), or any perplexity/FLOP gain disappears when utilization and dropped tokens are matched.",
 "math_summary": "The paper defines the average survival time of a location \\(\\mathbf r\\) on facet class \\(\\alpha\\) and finite domain \\(\\mathcal D\\) as \\(\\tau_\\alpha(\\mathbf r;\\mathcal D)=\\int_0^\\infty \\exp[-N_\\alpha(\\mathbf r,t;\\mathcal D)]\\,dt\\). Here \\(N_\\alpha(\\mathbf r,t;\\mathcal D)\\) is the cumulative expected coverage or nucleation hazard by time \\(t\\), and \\(\\exp[-N_\\alpha]\\) is the probability that \\(\\mathbf r\\) remains uncovered. The transferable object is the finite-domain survival integral: boundary-limited coverage produces a characteristic completion time that depends on the instantaneous domain size. For an MoE router, expert \\(i\\) has normalized occupancy \\(s_i\\in[0,1]\\), and we define a differentiable proxy hazard \\(N_i(t;s_i)=\\lambda_i(s_i)t^p\\), with \\(p\u003e0\\) and \\(\\lambda_i(s_i)=\\lambda_0(s_i+\\epsilon)^\\beta\\). The resulting survival time is \\(\\tau_i(s_i)=\\int_0^\\infty e^{-\\lambda_i(s_i)t^p}dt=\\Gamma(1/p)/(p\\lambda_i(s_i)^{1/p})\\). Set the expert's allocation-growth velocity to \\(v_i(s_i)=1/\\tau_i(s_i)=c_p\\lambda_i(s_i)^{1/p}\\), where \\(c_p=p/\\Gamma(1/p)\\). For a continuous occupancy dynamics \\(\\dot s_i=v_i(s_i)-\\bar v\\), a perturbation \\(\\delta_i\\) around equal occupancy obeys approximately \\(\\dot\\delta_i=v'(s)\\delta_i\\); therefore \\(\\beta\u003e0\\) gives local asymmetry amplification and \\(\\beta\u003c0\\) gives balancing feedback. The neural implementation should use a bounded discrete approximation of this dynamics rather than allowing unbounded expert capacity.",
 "math_tags": [
  "stochastic-processes",
  "dynamical-systems",
  "probability",
  "geometry"
 ],
 "ml_areas": [
  "moe-routing",
  "training-dynamics",
  "transformer"
 ],
 "paper": {
  "arxiv_id": "2609.00145",
  "arxiv_url": "https://arxiv.org/abs/2609.00145",
  "summary_what_math_gives_to_ml": "The paper gives a constructive mechanism for deterministic amplification of arbitrarily small asymmetries: a finite-domain growth event has a survival time determined by an integrated coverage hazard, and the resulting facet velocity depends on the current facet size. This suggests a neural module in which routing, token allocation, or active-computation regions grow at rates determined by their current occupancy, creating controllable positive or negative feedback between usage and future capacity. The most direct experiment is a differentiable size-dependent mixture-of-experts router, comparing occupancy-independent routing against a survival-time-derived growth rate while measuring whether tiny initial load perturbations amplify according to the predicted variational dynamics. The transfer is speculative and should be treated as a dynamical-systems-inspired adaptive routing mechanism rather than as a drop-in optimizer.",
  "title": "Size-Dependent Growth Rates Amplify Infinitesimal Asymmetry in Nanocrystals",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 6,
  "usefulness": 5
 },
 "solves": [
  "scalability",
  "accuracy",
  "stability"
 ],
 "title": "Finite-domain survival-time MoE router",
 "url": "https://synthcore.org/idea/2963/finite-domain-survival-time-moe-router",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}