Hermite compound-Poisson feature noise
Implementation & benchmark of arXiv:2608.16272 — Complete asymptotic expansion for a Durrmeyer variant of operators based on Hermite polynomials
Source paper: Complete asymptotic expansion for a Durrmeyer variant of operators based on Hermite polynomials arXiv:2608.16272 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace ordinary additive or multiplicative activation noise with a nonnegative count-valued perturbation generated by the Hermite operator kernel. For a nonnegative feature x, sample an integer N whose distribution is exactly the operator's weight sequence and feed N/n to the next layer; the parameter alpha controls an additional even-jump component and therefore changes the noise geometry independently of the ordinary Poisson component.
Formulas
Mathematical statement
The paper defines the positive linear operator $$(G_{n,\alpha}f)(x)=e^{-(nx+\alpha x^{2})}\sum_{\nu=0}^{\infty}\frac{x^{\nu}}{\nu!}H_{\nu}(n,\alpha)f\left(\frac{\nu}{n}\right),$$ where $n\in\mathbb{N}$, $\alpha\ge 0$, and $x\ge 0$, with $$H_{\nu}(n,\alpha)=\nu!\sum_{j=0}^{\lfloor \nu/2\rfloor}\frac{n^{\nu-2j}\alpha^j}{(\nu-2j)!j!}.$$ The exponential generating function is $$\sum_{\nu=0}^{\infty}\frac{x^{\nu}}{\nu!}H_{\nu}(n,\alpha)=\exp(nx+\alpha x^2).$$ Normalizing the coefficients gives the probability generating function $$\sum_{\nu=0}^{\infty}p_\nu(x)z^\nu=\exp\left(nx(z-1)+\alpha x^2(z^2-1)\right),$$ so the random index is $N=P+2Q$, with independent $P\sim\operatorname{Poisson}(nx)$ and $Q\sim\operatorname{Poisson}(\alpha x^2)$. Therefore $G_{n,\alpha}f(x)=\mathbb{E}[f(N/n)]$, $\mathbb{E}[N/n]=x+2\alpha x^2/n$, and $\operatorname{Var}(N/n)=x/n+4\alpha x^2/n^2$. The mean-preserving adaptation $Y=(N-2\alpha x^2)/n$ satisfies $\mathbb{E}[Y]=x$ while retaining the same variance.
Implementation notes
Integrate the method at the input of one or two MLP blocks, after enforcing nonnegative features with Softplus or ReLU. For a feature tensor X, choose integer n and nonnegative alpha; sample elementwise P~Poisson(n*X) and Q~Poisson(alpha*X^2), set N=P+2*Q, and use Y=(N-2*alpha*X^2)/n during training. The subtraction makes the stochastic layer unbiased because E[Y]=X; optionally replace it with Y=N/n if a positive mean shift is desired. Pseudocode is: X=softplus(h); P=Poisson(n*X); Q=Poisson(alpha*X*X); Y=(P+2*Q-2*alpha*X*X)/n; h_next=Linear(Y). The exact mathematical quantities are the compound-Poisson sampling law and its moments. Estimate alpha by validation search or learn alpha=softplus(a) per layer; use fixed n in {4,16,64}. Since Poisson sampling is not pathwise differentiable, start with a straight-through estimator: use the sampled Y in the forward pass and the derivative of the mean-preserving Gaussian proxy X + sqrt(X/n+4*alpha*X^2/n^2)*epsilon in the backward pass. A simpler ablation can use the deterministic expectation Y=X to verify the implementation. The first cheap experiment is a 3-layer MLP on MNIST or Fashion-MNIST, comparing this layer against dropout and Gaussian noise at matched activation variance. Sweep alpha in {0,0.25,1} and n in {4,16,64}; measure test accuracy, calibration error, gradient-norm variance, and train/test loss gap. Success means equal-or-better test accuracy with a smaller train/test gap or lower gradient variance at matched forward noise variance.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.