Heterogeneity-Preserving Router Coarse-Graining

Apply this to a router, recurrent state-space model, or graph neural network whose behavior varies across identifiable environments such as token topic, graph community, sequence regime, or image class. Define M equal to 8 through 32 environment clusters using frozen embeddings or known metadata. During a calibration pass, collect expert occupancy and transition statistics separately for each environment, producing K_i^(z) and delta_i^(z). Fit both a heterogeneous router with environment-conditioned logits and an averaged router with one global logit correction. For every expert pair, estimate the single-expert equilibrium by iterating the discrete version of the coarse-grained dynamics with step size h. Compute the rare-expert Jacobian L and estimate its dominant eigenvalue using Arnoldi or power iteration. Initially use the eigenvalue only for architecture selection or a stop-gradient hinge penalty. The paper contributes the continuous-versus-finite habitat persistence mechanism; clusters and neural kernels are empirical adaptations. The first cheap experiment is a two-expert CIFAR-10 or WikiText-2 model with two deliberately distinct regimes, comparing global routing, environment-conditioned routing, and random conditioning at equal parameter count. The prediction is that averaging lowers at least one invasion eigenvalue and that collapse begins when the averaged estimate crosses zero. Test whether the observed collapse threshold tracks the eigenvalue-zero crossing within 20 percent and whether preserving heterogeneity improves minority-regime loss without increasing total compute.