Grunbaum Entropy-Preserving Router

Integrate this into a binary MoE router or a binary hierarchical latent partition. Given token features x in a batch, compute a scalar projection z=w^T x, use a detached batch mean mu=mean(z), and route tokens with z<=mu to expert 0 and the rest to expert 1. Keep the hard routing for the first MVP; a straight-through sigmoid around z-mu can be used if w must receive gradients. Every K steps, estimate the parent and child differential entropies from z using a fixed-bin histogram with a small additive pseudocount, or use the Gaussian proxy h_G=0.5 log(2 pi e sigma^2) as a low-variance first experiment. Compute C_G=e/(e-1)*[-(1/e)log(1/e)-(1-1/e)log(1-1/e)], then add lambda_def times the displayed entropy-deficit barrier. Also add a separate parent entropy floor, such as lambda_floor*max(0,h_min-h_parent)^2, because the deficit constraint alone permits both parent and child entropies to collapse together. Maintain an exponential moving average of mu for routing stability and skip the entropy term when either child has fewer than a minimum number of tokens. The mathematical quantities are the mean cut, the 1/e mass bound, and C_G; entropy estimators, smoothing, and regularization weights are empirical choices. First test on a 4-expert sparse MoE by recursively applying mean cuts, using a small Transformer on WikiText-2 or CIFAR-10 patches. Compare against standard top-1 routing at equal parameter count and FLOPs. Measure validation loss, dead-expert frequency, per-expert token counts, projected-feature entropy, and seed-to-seed training variance. Success means fewer dead experts and more stable training with equal or better validation loss and no meaningful throughput loss.