{
 "artifacts": null,
 "category": "regularization",
 "description": "Regularize a hard top-1 or top-k router so its selected expert set remains stable under small logit and capacity-mask perturbations. Combine perturbation-based route consistency with a positive gap between selected and unselected experts, because stable objective values alone do not guarantee stable argmax solutions.",
 "formulas_latex": [
  "$$f^{k}\\overset{c}{\\rightarrow}f\\Longleftrightarrow h\\text{-}\\limsup_{k}f^{k}\\leq f\\leq e\\text{-}\\liminf_{k}f^{k}\\Longleftrightarrow f^{k}\\overset{h}{\\rightarrow}f\\text{ and }f^{k}\\overset{e}{\\rightarrow}f.$$",
  "$$\\Psi^{k}\\overset{g}{\\rightarrow}\\Psi\\Longleftrightarrow\\operatorname{gph}\\Psi^{k}\\rightarrow\\operatorname{gph}\\Psi\\Longleftrightarrow\\begin{cases}\\Psi\\subset g\\text{-}\\liminf_{k}\\Psi^{k},\\\\g\\text{-}\\limsup_{k}\\Psi^{k}\\subset\\Psi.\\end{cases}$$",
  "$$u^{k}(x,y)=\\frac{1}{k}|x|,\\qquad u(x,y)\\equiv0,\\qquad\\Phi^{k}(y)=\\Phi(y)=[-1,1],\\qquad S^{k}(y)=\\{-1,1\\},\\quad S(y)=[-1,1].$$",
  "$$\\mathcal L_{\\mathrm{route}}=\\frac{1}{BJ}\\sum_{b=1}^{B}\\sum_{j=1}^{J}\\left(1-\\frac{|R_{b}^{(j)}\\cap R_{b}^{(0)}|}{K}\\right)+\\mu\\frac{1}{B}\\sum_{b=1}^{B}\\max\\left(0,\\gamma-\\left[\\min_{i\\in R_b^{(0)}}z_{b,i}-\\max_{j\\notin R_b^{(0)}}z_{b,j}\\right]\\right).$$"
 ],
 "id": 182,
 "implementation": "(1) Integrate the method at the router logits of a Switch-style or top-k Mixture-of-Experts Transformer. For token representation $h_b$, compute $z_b=W h_b\\in\\mathbb R^E$, apply the feasible capacity mask $M_b$, and define the clean selected set $R_b^{(0)}=\\operatorname{TopK}(z_b,M_b)$. Keep the forward routing operation unchanged initially. (2) For each batch, generate $J=2$ or $4$ perturbations $z_b^{(j)}=z_b+\\epsilon_b^{(j)}$, with $\\epsilon_b^{(j)}\\sim\\mathcal N(0,\\sigma^2I)$, and optionally remove one randomly chosen feasible expert from $M_b$ to model capacity or availability changes. Recompute $R_b^{(j)}=\\operatorname{TopK}(z_b^{(j)},M_b^{(j)})$. Add the displayed route-consistency loss; use the exact set-overlap expression as a detached discrete diagnostic or replace it during backpropagation with a soft top-k relaxation. Add the margin hinge, where $\\gamma$ is the desired score gap. (3) The paper supplies the mathematical reason for monitoring solution mappings rather than only values. Estimate $\\sigma$ from router-logit standard deviation using multipliers $10^{-3}$, $10^{-2}$, and $10^{-1}$, and sweep $\\gamma$ over $0.01$, $0.05$, and $0.1$ times that standard deviation. Compute route-flip rate, selected-set agreement, expert-load coefficient of variation, overflow rate, and clean validation loss. (4) First experiment: train a 4-layer Transformer with 8 experts on WikiText-2 or a small synthetic long-tailed classification dataset, comparing ordinary top-k routing with the regularized router at equal FLOPs and parameter count. A successful result is at least a twofold reduction in route flips and lower overflow under perturbation, with unchanged or improved perplexity/accuracy and no harmful expert-collapse or load imbalance.",
 "math_summary": "The paper states that continuous convergence of objectives is equivalent to simultaneous hypo- and epi-convergence: $$f^{k}\\overset{c}{\\rightarrow}f\\Longleftrightarrow h\\text{-}\\limsup_{k}f^{k}\\leq f\\leq e\\text{-}\\liminf_{k}f^{k}\\Longleftrightarrow f^{k}\\overset{h}{\\rightarrow}f\\text{ and }f^{k}\\overset{e}{\\rightarrow}f.$$ For set-valued mappings, graphical convergence satisfies $$\\Psi^{k}\\overset{g}{\\rightarrow}\\Psi\\Longleftrightarrow\\operatorname{gph}\\Psi^{k}\\rightarrow\\operatorname{gph}\\Psi\\Longleftrightarrow\\Psi\\subset g\\text{-}\\liminf_{k}\\Psi^{k},\\quad g\\text{-}\\limsup_{k}\\Psi^{k}\\subset\\Psi.$$ The paper explicitly demonstrates failure of inner solution stability with $$u^{k}(x,y)=\\frac{1}{k}|x|,\\quad u(x,y)\\equiv0,\\quad\\Phi^{k}(y)=\\Phi(y)=[-1,1],$$ for which the solution sets are $S^{k}(y)=\\{-1,1\\}$ while $S(y)=[-1,1]$. Here $u^{k}$ is a perturbed objective, $\\Phi^{k}$ is a feasible-set mapping, $v^{k}(y)=\\max_{x\\in\\Phi^{k}(y)}u^{k}(x,y)$ is the value function, and $S^{k}(y)=\\arg\\max_{x\\in\\Phi^{k}(y)}u^{k}(x,y)$ is the solution mapping. The neural surrogate is to enforce approximate uniqueness through a margin $\\gamma\u003e0$: the selected score must exceed every feasible competitor by at least $\\gamma$.",
 "math_tags": [
  "optimization",
  "topology",
  "set-valued-analysis"
 ],
 "ml_areas": [
  "moe-routing",
  "transformer",
  "regularization",
  "training-dynamics"
 ],
 "paper": {
  "arxiv_id": "2608.25789",
  "arxiv_url": "https://arxiv.org/abs/2608.25789",
  "summary_what_math_gives_to_ml": "The paper develops constructive stability criteria for value functions and argmax solution mappings under simultaneous perturbations of objectives and feasible sets. Its most transferable asset is the distinction between value stability and optimizer-set stability: convergent objective values do not guarantee that selected solutions remain stable, and inner convergence can fail even under continuous convergence. This directly applies to hard top-k routing, attention sparsification, and discrete action heads. A practical adaptation is to train routers with perturbation consistency and explicit selection margins, then test graph stability through route agreement under logit and mask perturbations.",
  "title": "Sequential Stability of the Value Function and the Solution Mapping in Berge's Maximum Theorem via Variational Convergence",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 5,
  "usefulness": 5
 },
 "solves": [
  "stability",
  "accuracy",
  "generalization"
 ],
 "title": "Graph-stable hard routing",
 "url": "https://synthcore.org/idea/182/graph-stable-hard-routing",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}