# Lorentzian coefficient regularization - ID: 2174 - Canonical URL: https://synthcore.org/idea/2174/lorentzian-coefficient-regularization - API JSON: https://synthcore.org/api/idea/2174.json - API Markdown: https://synthcore.org/api/idea/2174.md - Verification status: unverified - Source: [arXiv:2608.12266](https://arxiv.org/abs/2608.12266) - Category: regularization - Solves: stability, generalization, accuracy - ML areas: moe-routing, attention, loss, regularization - Math tags: combinatorics, convex-analysis, linear-algebra, algebra - Ratings: usefulness 6/10; difficulty 6/10; novelty 8/10 ## Idea description Represent a nonnegative neural output as a homogeneous polynomial with coefficients indexed by count vectors, and penalize violations of the Lorentzian Hessian signature after factorial normalization. Add an M-convex support penalty so mass can move between coordinates through valid exchange operations rather than forming disconnected or brittle coefficient patterns. ## Mathematical statement The paper defines the normalization operator for f(x)=\sum_{\alpha\in\mathbb{Z}_{\geq0}^n}c_\alpha x^\alpha by \mathcal{N}(f)=\sum_\alpha(c_\alpha/\alpha!)x^\alpha, where \alpha=(\alpha_1,\ldots,\alpha_n), x^\alpha=\prod_i x_i^{\alpha_i}, and \alpha!=\prod_i\alpha_i!. Its support is \operatorname{supp}(f)=\{\alpha:c_\alpha eq0\}. A support J is M-convex when for any \alpha,\beta\in J with \alpha_i>\beta_i, there exists j with \alpha_j<\beta_j such that both exchanged vectors \alpha-\mathbf e_i+\mathbf e_j and \beta-\mathbf e_j+\mathbf e_i remain in J; \mathbf e_i is the i-th standard basis vector. The Lorentzian characterization can be operationalized for a homogeneous degree-d polynomial g=\mathcal N(f): for every multi-index \gamma with |\gamma|=d-2, the Hessian H_\gamma= abla^2(\partial^\gamma g) has at most one positive eigenvalue, while \operatorname{supp}(g) is M-convex. Here \partial^\gamma=\prod_i\partial_{x_i}^{\gamma_i}, and |\gamma|=\sum_i\gamma_i. Penalizing the second-largest positive eigenvalue transfers this concrete signature condition into a differentiable neural regularizer. ## Key formulas - $$\mathcal{N}(f)=\sum_{\alpha\in\mathbb{Z}_{\geq 0}^{n}}\frac{c_{\alpha}}{\alpha!}x^{\alpha},\qquad x^\alpha=\prod_{i=1}^n x_i^{\alpha_i},\qquad \alpha!=\prod_{i=1}^n\alpha_i!.$$ - $$\operatorname{supp}(f)=\{\alpha\in\mathbb{Z}_{\geq0}^{n}:c_\alpha\neq0\}.$$ - $$\alpha,\beta\in J,\ \alpha_i>\beta_i\ \Longrightarrow\ \exists j:\ \alpha_j<\beta_j,\quad \alpha-\mathbf e_i+\mathbf e_j\in J,\quad \beta-\mathbf e_j+\mathbf e_i\in J.$$ - $$\mathcal{L}_{\mathrm{Lor}}(\theta)=\mathbb{E}_{\gamma:\,|\gamma|=d-2}\left[\max\{0,\lambda_2(H_\gamma)\}^2\right],\qquad H_\gamma=\nabla^2\!\left(\partial^\gamma\mathcal N(f_\theta)\right),$$ $$\mathcal{L}_{\mathrm{total}}=\mathcal{L}_{\mathrm{task}}+\lambda_{\mathrm{Lor}}\mathcal{L}_{\mathrm{Lor}}+\lambda_{M}\mathcal{L}_{M}.$$ ## Implementation notes Integrate this at the output head of a small mixture-count or routing model. Let the network emit logits u_alpha for a fixed finite set A of multi-indices alpha with |alpha|=d; define nonnegative coefficients c_alpha=softplus(u_alpha)+epsilon and normalized polynomial coefficients g_alpha=c_alpha/alpha!. For each minibatch, sample K multi-indices gamma with |gamma|=d-2. Construct the degree-two polynomial q_gamma(x)=partial^gamma g(x) analytically from the coefficient table, then form its n-by-n Hessian H_gamma at x=1, with H_gamma[i,j]=partial_{x_i}partial_{x_j}q_gamma(1). Compute the two largest eigenvalues using a symmetric eigensolver for n<=32, or estimate lambda_2 with two-step Lanczos for larger n, and add max(0,lambda_2)^2 to the task loss. For the support term, use a soft support mask m_alpha=sigmoid((c_alpha-tau)/temperature). Sample pairs alpha,beta and coordinates i,j satisfying alpha_i>beta_i and alpha_j