✗ Failed on benchmark 2026

Firmly Nonexpansive Convex-Gradient Denoiser

Implementation & benchmark of arXiv:2607.26864 — A Relaxed Gradient Step Denoiser for Splitting Methods in Poisson Inverse Problems

Usefulness7/10

Difficulty5/10

Novelty7/10

Architecture Stability Generalization Accuracy Cnn Training Regularization Convex analysis Optimization Dynamical systems

Source paper: A Relaxed Gradient Step Denoiser for Splitting Methods in Poisson Inverse Problems arXiv:2607.26864 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace an unconstrained image denoiser or refinement block by a gradient step on an input-convex neural potential. The resulting map has a verifiable nonexpansiveness guarantee when the potential is convex and its gradient is sufficiently smooth, reducing error amplification across repeated applications and making the module safer under distribution shift.

Formulas

$\mathbf{g}=\mathcal{N}(\mathbf{H}\mathbf{x}^{\star}+b),\qquad \mathbf{H}_{i,j}\geq 0,\quad \mathbf{H}^{\top}\mathbf{1}=\mathbf{1},\quad b\geq 0.$

$D_{\gamma}(x)=x-\gamma\nabla\phi_{\theta}(x),\qquad D_{\gamma,\rho}(x)=(1-\rho)x+\rho D_{\gamma}(x)=x-\rho\gamma\nabla\phi_{\theta}(x).$

$\langle \nabla\phi_{\theta}(x)-\nabla\phi_{\theta}(y),x-y\rangle\geq \frac{1}{L}\left\|\nabla\phi_{\theta}(x)-\nabla\phi_{\theta}(y)\right\|^{2},\qquad 0<\rho\gamma\leq \frac{1}{L}.$

$\left\|D_{\gamma,\rho}(x)-D_{\gamma,\rho}(y)\right\|^{2}\leq\left\langle D_{\gamma,\rho}(x)-D_{\gamma,\rho}(y),x-y\right\rangle.$

Mathematical statement

The extracted paper models Poisson observations as $\mathbf{g}=\mathcal{N}(\mathbf{H}\mathbf{x}^{\star}+b)$, with nonnegative blur operator $\mathbf{H}$ and background $b\geq 0$. The transferable construction is $D_{\gamma}(x)=x-\gamma\nabla\phi_{\theta}(x)$, where $x\in\mathbb{R}^{n}$ is the feature or image vector, $\phi_{\theta}:\mathbb{R}^{n}\to\mathbb{R}$ is convex in $x$, and $\gamma>0$ is a step size. Assume $\nabla\phi_{\theta}$ is $L$-Lipschitz, meaning $\|\nabla\phi_{\theta}(x)-\nabla\phi_{\theta}(y)\|\leq L\|x-y\|$ for all relevant $x,y$. Convexity plus smoothness gives the cocoercivity inequality $\langle \nabla\phi_{\theta}(x)-\nabla\phi_{\theta}(y),x-y\rangle\geq L^{-1}\|\nabla\phi_{\theta}(x)-\nabla\phi_{\theta}(y)\|^{2}$. Consequently, for $0<\gamma\leq 1/L$, $D_{\gamma}$ is firmly nonexpansive because $\|D_{\gamma}(x)-D_{\gamma}(y)\|^{2}\leq\langle D_{\gamma}(x)-D_{\gamma}(y),x-y\rangle$. A relaxed version $D_{\gamma,\rho}(x)=(1-\rho)x+\rho D_{\gamma}(x)=x-\rho\gamma\nabla\phi_{\theta}(x)$ uses effective step size $\rho\gamma$; choosing $0<\rho\gamma\leq 1/L$ preserves the same guarantee. Implement $\phi_{\theta}$ as an input-convex neural network by constraining hidden-to-hidden weights to be elementwise nonnegative and using convex nondecreasing activations, optionally adding a positive quadratic term to improve coercivity.

Implementation notes

Integrate this as a replacement for a residual CNN denoiser in a repeated refinement network, plug-and-play ADMM loop, or diffusion-style restoration step. The tensor entering the block is an image or feature map $x$; flatten only for notation, while implementing $\phi_{\theta}$ with convolutional layers. Build an ICNN potential using nonnegative hidden-to-hidden convolution weights, convex nondecreasing activations such as ReLU or softplus, and an optional term $\frac{\mu}{2}\|x\|^{2}$ with $\mu\geq 0$. Compute the denoiser by automatic differentiation: v = grad(phi_theta(x), x); x_new = x - rho*gamma*v. Use this loop: (1) x <- input; (2) phi <- ICNN(x); (3) v <- d phi / d x; (4) x <- x - rho*gamma*v; (5) repeat K times or feed x_new to the next reconstruction stage. Train on clean/noisy image pairs with a denoising loss such as $\|D(x_{noisy})-x_{clean}\|_2^2$, plus a small gradient penalty to prevent exploding slopes. The mathematical guarantee is computed from convex architectural constraints and a bound $L$; estimate $L$ by power iteration on Jacobian-vector products, $L\approx\max_j\|J_{\nabla\phi}(u_j)\|_2$, over validation samples, or use a conservative product-of-layer-norm bound. Set $\rho\gamma=0.9/L$ and compare against an unconstrained DnCNN residual block and a spectrally normalized block. First test on $32\times32$ MNIST or CIFAR-10 images with Gaussian and Poisson corruption, using 5--10 repeated applications. Success means lower worst-case amplification $\|x_K-y_K\|/\|x_0-y_0\|$, no exploding iterates under stronger corruption than seen in training, and competitive PSNR at equal parameter count; for a PnP test, also measure reconstruction PSNR and sensitivity across a 10x range of ADMM penalties.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built and ran a reproducible quadratic ICNN-special-case verification plus a small learned denoising comparison. The predicted firm-nonexpansiveness boundary was confirmed at alpha*L=1.00 passing and 1.01 failing; the repeated-update divergence boundary was confirmed at alpha*L=2.00, with 2.01 diverging; and the K-step contraction sweep at alpha*L=0.9 exactly matched the spectral prediction. The learned convex-gradient denoiser also had lower test MSE and lower 10-step paired sensitivity than the unconstrained residual baseline, although this is only a small illustrative signal.

Agent confidence: 9/10

Baseline: test MSE 0.27105; paired 10-step sensitivity 0.40893
Idea: test MSE 0.24192; paired 10-step sensitivity 0.33858; learned Hessian maximum eigenvalue 4.0844; firm boundary last pass alphaL=1.0 and first fail 1.01; divergence first observed alphaL=2.01; alphaL=0.9 repeated ratios for K=[1,2,5,10,20] were [0.96625,0.93364,0.84226,0.70941,0.50326], exactly matching prediction

Limitations: The learned model is a quadratic PSD-potential special case rather than a convolutional multi-layer ICNN. The baseline is an affine residual map rather than a full DnCNN or spectrally normalized CNN. No MNIST/CIFAR, Poisson, PnP-ADMM, distribution-shift, FLOP-matched, or multi-seed evaluation was performed.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.26864) Sep 2, 2026

No significant difference vs. the tuned baseline (+11.1%, p=1, wins 7 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: custom_matched_refinement_rnn
Paired seeds: 8
Baseline mean: 0.0001
Idea mean: 0.0001
Effect (Δ): 0 (+11.1%; negative = idea better)
Wins: 7 / 8 paired seeds
p-value: 1 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±119.8%
Mechanism: Not confirmed ✗
Practical effect: Inconclusive
Baseline tuning: swept over 12 configs

Limitations:

The built-in dynamics benchmark evaluates one-step pendulum prediction rather than a long-horizon rollout objective. No image denoising, Poisson/PnP, distribution-shift, or FLOP-matched evaluation was run; the ICNN is a compact feature-space potential rather than a convolutional image ICNN.

How to run: python3 bench_stage2.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 10/10

Implemented the convex-gradient refinement RNN and matched unconstrained residual RNN on the registered dynamics track using bench.train_model, paired seeds, and a shared hyperparameter union. The baseline full mean test MSE was 0.0001078151 versus 0.0001197831 for the idea, with paired delta +0.0000119680 and permutation p=1.0. The mechanism signature was not confirmed because trained-model output sensitivity was slightly worse for the idea on average.

Baseline: Best lr=0.003, step=0.25; full 8-seed mean MSE=0.00010781512173707597
Comparison: delta_mean=0.00001196796006297518; p_value=1.0; idea_wins=7/8; verdict=no significant win
Idea: lr=0.003, step=0.25; full 8-seed mean MSE=0.00011978308180005115

Benchmark result