Lorentzian coefficient router

Integrate this into a small MoE router or attention block rather than a full Transformer. Let K be 4–8 experts or channels and choose polynomial degree d=3 or 4. Store unconstrained parameters u_alpha for every multi-index with |alpha|=d, and set c_alpha=softplus(u_alpha) so coefficients remain nonnegative. For each token representation h, produce positive variables x=softplus(W h+b)+epsilon. Evaluate monomials z_alpha=c_alpha x^alpha/alpha!, then map z to K expert logits with a learned matrix or use grouped monomial scores directly; apply softmax for routing. During training, sample a small subset B of multi-indices beta satisfying |beta|=d-2. Differentiate F analytically or with automatic differentiation, form H_beta at xi=ones(K) or at the normalized batch-average x, compute its eigenvalues, and add R_Lor to the task loss. The paper-derived elements are factorial normalization, tableau-based coefficients, and the one-positive-eigenvalue target; Hessians and eigenvalues are estimated numerically. For initialization, enumerate semistandard tableaux for a tiny shape such as lambda/mu=(3,2)/(1), accumulate coefficients by content, normalize by alpha!, and initialize u_alpha with softplus_inverse(c_alpha+epsilon). First compare a 4-expert polynomial router with a standard softmax router on CIFAR-100 or a small Switch-MLP language model, matching parameter count and FLOPs. Track task loss, validation accuracy or perplexity, expert-load entropy, routing-collapse rate, gradient variance, and the fraction of sampled Hessian slices with a second positive eigenvalue. Success means smoother loss curves, fewer dead or overloaded experts, and equal-or-better accuracy at equal compute.