Unverified Re-invented 2026

Certified nonconvex p-shrinkage

Implementation & benchmark of arXiv:2606.31951 — Explicit Series and a Certified Hybrid Evaluator for the $\ell_p$ Proximity Operator for $0<p<1$

Usefulness6/10
Difficulty4/10
Novelty6/10

Source paper: Explicit Series and a Certified Hybrid Evaluator for the $\ell_p$ Proximity Operator for $0<p<1$ arXiv:2606.31951 · analyzed Aug 29, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace soft-thresholding or an approximate smooth l_p penalty with the scalar proximal operator of lambda|x|^p for 0<p<1. Apply the exact threshold first, then evaluate the larger positive stationary root using the paper's Lagrange-inversion series when it is well-conditioned and safeguarded bisection near the threshold.

Formulas

$$F(x;y)=\frac{1}{2}(x-y)^2+\lambda|x|^p,\qquad \operatorname{prox}_{\lambda|\cdot|^p}(y)\in\arg\min_x F(x;y).$$
$$h(u)=u+\lambda p u^{p-1}=|y|,\qquad u_0=[\lambda p(1-p)]^{\frac{1}{2-p}},\qquad u_{\mathrm{th}}=[2\lambda(1-p)]^{\frac{1}{2-p}},$$ $$r_{\mathrm{th}}=\frac{2-p}{2(1-p)}u_{\mathrm{th}},\qquad \operatorname{prox}_{\lambda|\cdot|^p}(y)=0\ \text{if }|y|<r_{\mathrm{th}}.$$
$$s=\lambda p|y|^{p-2},\qquad u=|y|\left[1-\sum_{n=1}^{N}c_n(q)s^n\right]+R_N,\quad q=1-p,\quad c_n(q)=\frac{\Gamma((q+1)n-1)}{n\,\Gamma(qn)\Gamma(n)},\quad c_1(q)=1.$$
$$\widehat{u}\in[u_0,|y|],\qquad |h(\widehat{u})-|y||\leq\varepsilon_{\mathrm{root}};\qquad x^+=\operatorname{sign}(y)\widehat{u}.$$

Mathematical statement

For a scalar input y, define P_{lambda,p}(y) as a global minimizer of F(x;y)=0.5(x-y)^2+lambda|x|^p, with lambda>0 and 0<p<1. By symmetry it is enough to use r=|y| and return sign(y) times a nonnegative solution. Any nonzero stationary point u>0 satisfies h(u)=u+lambda p u^{p-1}=r. The stationary equation has its minimum at u_0=[lambda p(1-p)]^{1/(2-p)}, and the global minimizer switches from zero at the threshold r_th=((2-p)/(2(1-p)))u_th, where u_th=[2lambda(1-p)]^{1/(2-p)}. Thus P_{lambda,p}(y)=0 for |y|<r_th; at equality either zero or sign(y)u_th is globally optimal; above the threshold choose the larger root u>u_0 of h(u)=|y|. For the larger root, set q=1-p and s=lambda p |y|^{p-2}. Lagrange-Burmann inversion gives u=|y|[1-s-sum_{n=2}^N c_n(q)s^n]+R_N, with c_n(q)=Gamma((q+1)n-1)/(n Gamma(qn)Gamma(n)); equivalently c_1(q)=1. The series is used only in its certified or empirically validated convergence region. Near threshold, bracket the larger root in [u_0, |y|] and use bisection or safeguarded Newton; the residual |h(u)-|y|| supplies an implementation certificate.

Implementation notes

Integrate this into a proximal-gradient optimizer for a selected parameter tensor, such as all weights of a small MLP or only the router weights of an MoE layer. At iteration t, compute the ordinary gradient step z=W_t-eta_t*grad_W L(W_t), then apply the scalar proximal map independently to every entry of z with lambda=eta_t*alpha, where alpha is the l_p regularization coefficient. For each z_i, let r=abs(z_i), compute u_th=[2*lambda*(1-p)]**(1/(2-p)) and r_th=((2-p)/(2*(1-p)))*u_th; if r<r_th set W_new_i=0. Otherwise compute s=lambda*p*r**(p-2). If s is below a configurable series limit, initialize u=r*(1-s-sum_{n=2}^N c_n*s**n), evaluating c_n with log-Gamma functions, and accept it only when abs(h(u)-r)<=epsilon. Otherwise set lo=u_0=[lambda*p*(1-p)]**(1/(2-p)) and hi=r, then perform 30-50 bisection iterations on h(u)-r for the larger-root branch; use Newton steps only when they stay inside the bracket. The mathematical quantities are the threshold, stationary-root equation, and series coefficients. Estimate truncation reliability by comparing orders N and 2N; certify root accuracy using residual and bracket width. Start with a 2-layer width-512 MLP on MNIST or CIFAR-10. Compare AdamW plus magnitude pruning, soft-threshold proximal l1, and direct subgradient l_p training at matched sparsity and FLOPs. Measure validation accuracy, exact zero fraction, proximal-call time, wall-clock time, and loss descent. Success means higher accuracy at the same zero fraction, or equal accuracy with fewer pruning operations, with no threshold oscillations and residuals below 1e-6.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.