A Layered Simplex Architecture for Large Alphabets

What the math gives to ML

The paper provides a simple way to construct priors with tunable concentration and heavy-tailed sparsity: multiply independent random simplex vectors coordinate-wise and renormalize. Its transferable asset is not the alphabet estimator itself, but the fact that multiplicative depth spreads prior mass across many effective complexity scales, with logarithmic depth producing a few dominant coordinates and many nearly-zero ones. This suggests using layered simplex factors as a principled prior for MoE routing, attention-head selection, output vocabulary biases, or other normalized neural gates. The first implementation should target MoE routers, comparing fixed-depth multiplicative gates against ordinary softmax routing at equal expert capacity.