Fast-Class MoE Router

Integrate this into the MoE router before expert selection. Let each token representation h produce pairwise scores a_xy for a small candidate set of experts. Assign every expert to a group g(x), construct rates r_gamma using the displayed rule, and form a generator Q with Q_xy=r_gamma(x,y) for x != y and Q_xx=-sum_{y != x} r_gamma(x,y). For a cheap differentiable approximation, use P=I+Q/c with c >= max_x(-Q_xx), then run 10-30 power iterations v <- normalize(vP) to estimate the stationary vector. Separately form the fast within-group generators by retaining only same-group edges, estimate each mu_M with the same iteration, and compute bar_r(M,M') using the effective-rate formula. Run power iteration again on the class-level generator to obtain bar_pi. Add the displayed L_stat to the task loss, using rho as the minibatch group-load vector. During routing, use logits a_x + beta log(bar_pi(g(x))+epsilon), then select top-k experts subject to capacity. Stop-gradient through stationary estimates initially; later test implicit or unrolled differentiation. Begin with a 12-expert, 4-group, top-2 MoE Transformer on C4 or a synthetic mixture of token domains. Compare against Switch-style routing and its standard load-balancing loss at equal FLOPs. Record perplexity, expert-load coefficient of variation, token overflow, routing changes between successive batches, and all-to-all communication. Success is lower overflow and routing churn at matched perplexity, or faster loss decrease without increased communication.