# Mean-Polynomial Positivity Head

- ID: 72
- Canonical URL: https://synthcore.org/idea/72/mean-polynomial-positivity-head
- API JSON: https://synthcore.org/api/idea/72.json
- API Markdown: https://synthcore.org/api/idea/72.md
- Verification status: unverified
- Source: [arXiv:2608.22739](https://arxiv.org/abs/2608.22739)
- Category: regularization
- Solves: accuracy, stability, generalization
- ML areas: loss, regularization, mlp
- Math tags: convex-analysis, approximation-theory, optimization
- Ratings: usefulness 5/10; difficulty 5/10; novelty 8/10

## Idea description

Parameterize a nonnegative neural penalty or energy function as a sum of weighted power-mean differences applied to polynomial features of the network representation. Each atom is globally nonnegative by the power-mean inequality, so the learned penalty cannot become negative or destabilize constrained training, while the cone can represent polynomials outside SOS-plus-nonnegative-circuit certificates.

## Mathematical statement

For positive weights w_i>0, arguments Y_i, and orders q>p>=1, the paper defines M_p(Y,w)=((sum_i w_iY_i^p)/(sum_i w_i))^(1/p) and M_{q,p}(Y,w)=M_q(Y,w)^lcm(q,p)-M_p(Y,w)^lcm(q,p). The weighted power-mean inequality gives M_q(Y,w)>=M_p(Y,w), hence M_{q,p}(Y,w)>=0 for nonnegative arguments. The cone M_{n,2d} consists of finite sums of these atoms with polynomial arguments L_i(x) of total degree 2d. A concrete square identity is M_{2,1}((u,0),(1,1))=u^2/4, showing that squares are included. For neural features h, use nonnegative arguments Y_i(h)=softplus(P_i(h)); then E(h)=sum_j a_j M_{q_j,p_j}(Y^(j)(h),w^(j))^ell_j is globally nonnegative when a_j>=0. The paper's Robinson-form separation indicates that this cone can be strictly more expressive than SOSONC.

## Key formulas

- $$M_p(Y,w)=\left(\frac{\sum_{i=1}^{m}w_iY_i^p}{\sum_{i=1}^{m}w_i}\right)^{1/p},\qquad M_{q,p}(Y,w)=M_q(Y,w)^{\operatorname{lcm}(q,p)}-M_p(Y,w)^{\operatorname{lcm}(q,p)},\quad q>p,$$
- $$M_q(Y,w)\ge M_p(Y,w)\quad\Longrightarrow\quad M_{q,p}(Y,w)\ge0,$$
- $$M_{2,1}((u,0),(1,1))=\frac{u^2}{2}-\left(\frac{u}{2}\right)^2=\frac{u^2}{4},$$
- $$E_\phi(h)=\sum_{j=1}^{J}a_j\left[M_{q_j}(Y^{(j)}(h),w^{(j)})^{\ell_j}-M_{p_j}(Y^{(j)}(h),w^{(j)})^{\ell_j}\right],\quad a_j=\operatorname{softplus}(\alpha_j),\quad \ell_j=\operatorname{lcm}(q_j,p_j).$$

## Implementation notes

Integrate the construction as a nonnegative energy or regularization head on top of an MLP representation h=f_theta(x), rather than modifying the main prediction logits. Choose J=8--32 atoms. For each atom j, create a small feature map P_j(h) producing m nonnegative arguments, set Y^(j)=softplus(P_j(h)), normalize weights with w^(j)=softmax(beta_j), and constrain the atom coefficient with a_j=softplus(alpha_j). Start with integer orders (q,p)=(2,1); later add (4,2) or (3,1) as an ablation. Compute `h=f_theta(x)`, then for each j compute `Y=softplus(P_j(h))`, `mq=(sum(w*Y**q)/sum(w))**(1/q)`, `mp=(sum(w*Y**p)/sum(w))**(1/p)`, and `atom=a_j*(mq**l-mp**l)`, where l=lcm(q,p). Sum atoms into E and optimize `loss=task_loss+lambda*E`. The positivity guarantee comes from the power-mean inequality; no positivity estimate is needed during training. Empirically monitor the smallest E on random and adversarially scaled inputs, gradient norms, approximation error, and runtime. First fit a two-dimensional nonnegative target polynomial, including a Robinson-like quartic, on a dense grid. Compare against an SOS Gram parameterization and a sum of nonnegative circuit polynomials with matched parameter count. Then insert each penalty into constrained regression or energy-based classification. Success means lower target approximation error at equal parameter count, no sampled negative energies, fewer gradient spikes, or fewer constraint violations. Record wall-clock overhead and memory because repeated fractional powers may erase the benefit.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.