# Lorentzian coefficient router

- ID: 2087
- Canonical URL: https://synthcore.org/idea/2087/lorentzian-coefficient-router
- API JSON: https://synthcore.org/api/idea/2087.json
- API Markdown: https://synthcore.org/api/idea/2087.md
- Verification status: unverified
- Source: [arXiv:2608.10516](https://arxiv.org/abs/2608.10516)
- Category: regularization
- Solves: stability, generalization, accuracy
- ML areas: moe-routing, attention, regularization
- Math tags: combinatorics, geometry, linear-algebra, optimization
- Ratings: usefulness 5/10; difficulty 6/10; novelty 7/10

## Idea description

Represent a small expert router or attention interaction by a homogeneous polynomial with nonnegative coefficients, then penalize violations of the Lorentzian Hessian signature on degree-two derivative slices. Initialize or warm-start the coefficient tensor from a normalized skew-Schur coefficient array, which the paper identifies as a realizable volume polynomial and therefore a structurally valid Lorentzian point.

## Mathematical statement

For a polynomial F(x)=\sum_{\alpha}c_{\alpha}x^{\alpha}, with multi-index \alpha=(\alpha_1,\ldots,\alpha_K), x^{\alpha}=\prod_i x_i^{\alpha_i}, and \alpha!=\prod_i\alpha_i!, the paper defines the normalized polynomial \mathcal{N}(F)=\sum_{\alpha}c_{\alpha}x^{\alpha}/\alpha!. For a skew shape \lambda/\mu, s_{\lambda/\mu}(x_1,\ldots,x_K)=\sum_T x^{\operatorname{cont}(T)}, where T ranges over semistandard tableaux and \operatorname{cont}(T) records the number of entries of each value. The paper states that \mathcal{N}(s_{\lambda/\mu}), and analogously normalized skew P/Q-functions, are realizable volume polynomials and hence Lorentzian. The implementable Lorentzian condition for a homogeneous degree-d polynomial is that, for every multi-index \beta with |\beta|=d-2, the Hessian H_\beta(x)=
abla^2(\partial^\beta F)(x), evaluated at a positive point x, has at most one positive eigenvalue. Equivalently, if eigenvalues are sorted in descending order, \lambda_2(H_\beta(x))\le 0. Here \partial^\beta=\prod_i\partial_{x_i}^{\beta_i}, d is the polynomial degree, and K is the number of routing variables.

## Key formulas

- $$s_{\lambda/\mu}(x_1,\ldots,x_K)=\sum_T x^{\operatorname{cont}(T)},\qquad x^{\operatorname{cont}(T)}=\prod_{i=1}^K x_i^{\operatorname{cont}_i(T)}$$
- $$\mathcal{N}(F):=\sum_{\alpha}c_{\alpha}\frac{x^{\alpha}}{\alpha!},\qquad \alpha!:=\prod_{i=1}^K\alpha_i!$$
- $$H_{\beta}(x):=\nabla_x^2\!\left(\partial^{\beta}F(x)\right),\qquad |\beta|=d-2,\qquad \lambda_2(H_{\beta}(x))\le 0$$
- $$\mathcal{R}_{\mathrm{Lor}}(F)=\frac{1}{|\mathcal{B}|}\sum_{\beta\in\mathcal{B}}\left[\max\!\left(0,\lambda_2\!\left(H_{\beta}(\xi)\right)\right)\right]^2+\rho\sum_{\alpha}\left[\max(0,-c_{\alpha})\right]^2$$

## Implementation notes

Integrate this into a small MoE router or attention block rather than a full Transformer. Let K be 4–8 experts or channels and choose polynomial degree d=3 or 4. Store unconstrained parameters u_alpha for every multi-index with |alpha|=d, and set c_alpha=softplus(u_alpha) so coefficients remain nonnegative. For each token representation h, produce positive variables x=softplus(W h+b)+epsilon. Evaluate monomials z_alpha=c_alpha x^alpha/alpha!, then map z to K expert logits with a learned matrix or use grouped monomial scores directly; apply softmax for routing. During training, sample a small subset B of multi-indices beta satisfying |beta|=d-2. Differentiate F analytically or with automatic differentiation, form H_beta at xi=ones(K) or at the normalized batch-average x, compute its eigenvalues, and add R_Lor to the task loss. The paper-derived elements are factorial normalization, tableau-based coefficients, and the one-positive-eigenvalue target; Hessians and eigenvalues are estimated numerically. For initialization, enumerate semistandard tableaux for a tiny shape such as lambda/mu=(3,2)/(1), accumulate coefficients by content, normalize by alpha!, and initialize u_alpha with softplus_inverse(c_alpha+epsilon). First compare a 4-expert polynomial router with a standard softmax router on CIFAR-100 or a small Switch-MLP language model, matching parameter count and FLOPs. Track task loss, validation accuracy or perplexity, expert-load entropy, routing-collapse rate, gradient variance, and the fraction of sampled Hessian slices with a second positive eigenvalue. Success means smoother loss curves, fewer dead or overloaded experts, and equal-or-better accuracy at equal compute.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.