Lorentzian interaction router

Integrate the module into a higher-order MoE router or low-rank attention bias. Let each token representation h produce z=softplus(Wh+b) in R^n. Store trainable logits theta_alpha for all multi-indices alpha with |alpha|=d, set c_alpha=softplus(theta_alpha), and compute the interaction score g(z)=log(epsilon+sum_alpha c_alpha z^alpha/alpha!). Add g(z) either to every expert logit through an expert-specific coefficient tensor or use it as a shared load-aware router bias. During each training step, sample B feature points z_b from the current minibatch and nonnegative directions u_{b,1},...,u_{b,d-2}; use normalized softplus random projections or fixed coordinate mixtures. Compute the directional derivative polynomial with automatic differentiation, form its n-by-n Hessian H_b, and add lambda_L times max(0,lambda_2(H_b))^2 to the task loss. For d=3, only one directional derivative is needed; for d=4, use two. The paper's mathematics supplies the factorial representation and the one-positive-eigenvalue criterion. Estimate lambda_2 with a symmetric eigensolver for n<=32, or approximate it using a differentiable eigensolver/power iteration. Test n=16,d=3 in a small Switch-style MoE on CIFAR-100 or WikiText-103, comparing a standard softmax router, an unconstrained positive polynomial router, and the Lorentzian-regularized router at equal FLOPs. Track validation loss, expert-load coefficient of variation, router entropy, gradient norm spikes, and training time. Success is lower load variance and fewer instability spikes with equal or better validation loss; reject it if the curvature penalty costs over 10% training time without improving stability.