Certified nonconvex p-shrinkage
Implementation & benchmark of arXiv:2606.31951 — Explicit Series and a Certified Hybrid Evaluator for the $\ell_p$ Proximity Operator for $0<p<1$
Source paper: Explicit Series and a Certified Hybrid Evaluator for the $\ell_p$ Proximity Operator for $0<p<1$ arXiv:2606.31951 ⓘ · analyzed Aug 29, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace soft-thresholding or an approximate smooth l_p penalty with the scalar proximal operator of lambda|x|^p for 0<p<1. Apply the exact threshold first, then evaluate the larger positive stationary root using the paper's Lagrange-inversion series when it is well-conditioned and safeguarded bisection near the threshold.
Formulas
Mathematical statement
For a scalar input y, define P_{lambda,p}(y) as a global minimizer of F(x;y)=0.5(x-y)^2+lambda|x|^p, with lambda>0 and 0<p<1. By symmetry it is enough to use r=|y| and return sign(y) times a nonnegative solution. Any nonzero stationary point u>0 satisfies h(u)=u+lambda p u^{p-1}=r. The stationary equation has its minimum at u_0=[lambda p(1-p)]^{1/(2-p)}, and the global minimizer switches from zero at the threshold r_th=((2-p)/(2(1-p)))u_th, where u_th=[2lambda(1-p)]^{1/(2-p)}. Thus P_{lambda,p}(y)=0 for |y|<r_th; at equality either zero or sign(y)u_th is globally optimal; above the threshold choose the larger root u>u_0 of h(u)=|y|. For the larger root, set q=1-p and s=lambda p |y|^{p-2}. Lagrange-Burmann inversion gives u=|y|[1-s-sum_{n=2}^N c_n(q)s^n]+R_N, with c_n(q)=Gamma((q+1)n-1)/(n Gamma(qn)Gamma(n)); equivalently c_1(q)=1. The series is used only in its certified or empirically validated convergence region. Near threshold, bracket the larger root in [u_0, |y|] and use bisection or safeguarded Newton; the residual |h(u)-|y|| supplies an implementation certificate.
Implementation notes
Integrate this into a proximal-gradient optimizer for a selected parameter tensor, such as all weights of a small MLP or only the router weights of an MoE layer. At iteration t, compute the ordinary gradient step z=W_t-eta_t*grad_W L(W_t), then apply the scalar proximal map independently to every entry of z with lambda=eta_t*alpha, where alpha is the l_p regularization coefficient. For each z_i, let r=abs(z_i), compute u_th=[2*lambda*(1-p)]**(1/(2-p)) and r_th=((2-p)/(2*(1-p)))*u_th; if r<r_th set W_new_i=0. Otherwise compute s=lambda*p*r**(p-2). If s is below a configurable series limit, initialize u=r*(1-s-sum_{n=2}^N c_n*s**n), evaluating c_n with log-Gamma functions, and accept it only when abs(h(u)-r)<=epsilon. Otherwise set lo=u_0=[lambda*p*(1-p)]**(1/(2-p)) and hi=r, then perform 30-50 bisection iterations on h(u)-r for the larger-root branch; use Newton steps only when they stay inside the bracket. The mathematical quantities are the threshold, stationary-root equation, and series coefficients. Estimate truncation reliability by comparing orders N and 2N; certify root accuracy using residual and bracket width. Start with a 2-layer width-512 MLP on MNIST or CIFAR-10. Compare AdamW plus magnitude pruning, soft-threshold proximal l1, and direct subgradient l_p training at matched sparsity and FLOPs. Measure validation accuracy, exact zero fraction, proximal-call time, wall-clock time, and loss descent. Success means higher accuracy at the same zero fraction, or equal accuracy with fewer pruning operations, with no threshold oscillations and residuals below 1e-6.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.