# Convex-Ordered Count Head

- ID: 1258
- Canonical URL: https://synthcore.org/idea/1258/convex-ordered-count-head
- API JSON: https://synthcore.org/api/idea/1258.json
- API Markdown: https://synthcore.org/api/idea/1258.md
- Verification status: unverified
- Source: [arXiv:2607.21931](https://arxiv.org/abs/2607.21931)
- Category: regularization
- Solves: accuracy, stability, generalization
- ML areas: loss, regularization, mlp
- Math tags: probability, convex-analysis, statistics
- Ratings: usefulness 5/10; difficulty 5/10; novelty 7/10

## Idea description

Equip a neural-network count head with a mean parameter and a dispersion parameter from the Conway-Maxwell-Poisson family, then enforce a mean-preserving convex-order relationship between predictions. This provides a principled way to make the predictive count distribution more or less tail-dispersed while retaining the same predicted mean, potentially improving calibration on overdispersed or underdispersed count data.

## Mathematical statement

The paper models the stationary count distribution as $\pi_m(\phi,\sigma)=Z(\phi,\sigma)^{-1}\phi^m/[m!\prod_{k=1}^m\sigma(k)]$ for $m\in\{0,1,2,\ldots\}$, where $\phi>0$ is the acquisition rate, $\sigma(k)>0$ is the death-rate sequence, and $Z(\phi,\sigma)$ normalizes the probability mass. Its key result states that if the ratio $\sigma_1(m)/\sigma_2(m)$ is non-increasing in $m$, then the corresponding distributions after reparameterization to the same mean are ordered in convex order, with the direction determined by the two sequence labels. Convex order is $X_1\succeq_{cx}X_2$ iff $\mathbb{E}X_1=\mathbb{E}X_2$ and $\mathbb{E}f(X_1)\geq\mathbb{E}f(X_2)$ for every convex function $f$ for which the expectations exist. For the Conway-Maxwell-Poisson family, choose $\sigma_\nu(k)=k^{\nu-1}$, giving $p_\nu(m\mid\lambda)=\lambda^m/[(m!)^\nu Z(\lambda,\nu)]$. Since $\sigma_{\nu_1}(k)/\sigma_{\nu_2}(k)=k^{\nu_1-\nu_2}$ is non-increasing when $\nu_1<\nu_2$, the lower-$\nu$ mean-matched CMP law is more dispersed in convex order than the higher-$\nu$ law. The network outputs a target mean $\mu>0$ and dispersion $\nu$, while solving for $\lambda(\mu,\nu)$ such that $\mathbb{E}_{p_\nu(\lambda)}[X]=\mu$.

## Key formulas

- $$\pi_m(\phi,\sigma)=\frac{1}{Z(\phi,\sigma)}\frac{\phi^m}{m!\prod_{k=1}^{m}\sigma(k)},\qquad m=0,1,2,\ldots$$
- $$X_1\succeq_{cx}X_2\ \Longleftrightarrow\ \mathbb{E}[X_1]=\mathbb{E}[X_2]\ \text{ and }\ \mathbb{E}[f(X_1)]\geq\mathbb{E}[f(X_2)]\ \text{for every convex }f$$
- $$p_\nu(m\mid\lambda)=\frac{\lambda^m}{(m!)^\nu Z(\lambda,\nu)},\qquad \frac{\sigma_{\nu_1}(k)}{\sigma_{\nu_2}(k)}=k^{\nu_1-\nu_2}$$
- $$\mathcal{L}= -\log p_\nu(y\mid\lambda)+\rho\,\Big[\widehat{\mathbb{E}}f(X)-\widehat{\mathbb{E}}f(X_{\mathrm{ref}})\Big]_+,\qquad \widehat{\mathbb{E}}f(X)=\sum_{m=0}^{M}f(m)p_\nu(m\mid\lambda)$$

## Implementation notes

Integrate this into the final scalar-output head for a count-regression network. Let the network output unconstrained values $a,b$; set $\mu=\operatorname{softplus}(a)+\epsilon$ and $\nu=\nu_{\min}+\operatorname{softplus}(b)$, with for example $\nu_{\min}=0.2$. For each example, solve for the CMP rate $\lambda$ whose mean is exactly $\mu$. Truncate the support at $M=\lceil\mu+8\sqrt{\mu+\mu^2}\rceil$, capped at a batch-wide maximum. Compute log-weights $w_m=m\log\lambda-\nu\log(m!)$, normalize with log-sum-exp, and estimate the mean and variance. Use Newton updates on $r=\log\lambda$: $r\leftarrow r-(\widehat{\mathbb{E}}X-\mu)/\widehat{\operatorname{Var}}(X)$, because the derivative of the CMP mean with respect to $\log\lambda$ equals its variance. Stop after 5--10 iterations or when the mean error is below $10^{-4}$. Train with the truncated negative log-likelihood. To impose a desired convex-order relation, choose a reference dispersion $\nu_{\rm ref}$, solve its rate to the same mean $\mu$, and compute convex test expectations for $f_1(m)=m^2$, $f_2(m)=(m-t)_+^2$ at several thresholds, and $f_3(m)=\exp(\alpha m)$ with small $\alpha$. If predictions must not be more dispersed than the reference, add $\rho[\mathbb{E}f(X)-\mathbb{E}f(X_{\rm ref})]_+$ for each test function. The theorem justifies the constraint for all convex functions when the death-rate ratio is monotone; the finite test set is an empirical approximation. First test on synthetic counts generated from CMP laws with means sampled from $[1,20]$ and $\nu\in\{0.5,1,2\}$, comparing Poisson, negative-binomial, unconstrained CMP, and regularized CMP heads on a two-layer MLP. Measure held-out NLL, mean bias, 90% interval coverage, tail-probability error, and convex-risk violations. Success means lower tail-calibration error and fewer catastrophic underpredictions at matched NLL, while preserving mean accuracy. Also numerically verify the ordering by evaluating all selected convex expectations over a grid of matched means before enabling the regularizer.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.