# Firmly Nonexpansive Convex-Gradient Denoiser

- ID: 1465
- Canonical URL: https://synthcore.org/idea/1465/firmly-nonexpansive-convex-gradient-denoiser
- API JSON: https://synthcore.org/api/idea/1465.json
- API Markdown: https://synthcore.org/api/idea/1465.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.26864](https://arxiv.org/abs/2607.26864)
- Category: architecture
- Solves: stability, generalization, accuracy
- ML areas: cnn, training, regularization
- Math tags: convex-analysis, optimization, dynamical-systems
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Replace an unconstrained image denoiser or refinement block by a gradient step on an input-convex neural potential. The resulting map has a verifiable nonexpansiveness guarantee when the potential is convex and its gradient is sufficiently smooth, reducing error amplification across repeated applications and making the module safer under distribution shift.

## Mathematical statement

The extracted paper models Poisson observations as $\mathbf{g}=\mathcal{N}(\mathbf{H}\mathbf{x}^{\star}+b)$, with nonnegative blur operator $\mathbf{H}$ and background $b\geq 0$. The transferable construction is $D_{\gamma}(x)=x-\gamma\nabla\phi_{\theta}(x)$, where $x\in\mathbb{R}^{n}$ is the feature or image vector, $\phi_{\theta}:\mathbb{R}^{n}\to\mathbb{R}$ is convex in $x$, and $\gamma>0$ is a step size. Assume $\nabla\phi_{\theta}$ is $L$-Lipschitz, meaning $\|\nabla\phi_{\theta}(x)-\nabla\phi_{\theta}(y)\|\leq L\|x-y\|$ for all relevant $x,y$. Convexity plus smoothness gives the cocoercivity inequality $\langle \nabla\phi_{\theta}(x)-\nabla\phi_{\theta}(y),x-y\rangle\geq L^{-1}\|\nabla\phi_{\theta}(x)-\nabla\phi_{\theta}(y)\|^{2}$. Consequently, for $0<\gamma\leq 1/L$, $D_{\gamma}$ is firmly nonexpansive because $\|D_{\gamma}(x)-D_{\gamma}(y)\|^{2}\leq\langle D_{\gamma}(x)-D_{\gamma}(y),x-y\rangle$. A relaxed version $D_{\gamma,\rho}(x)=(1-\rho)x+\rho D_{\gamma}(x)=x-\rho\gamma\nabla\phi_{\theta}(x)$ uses effective step size $\rho\gamma$; choosing $0<\rho\gamma\leq 1/L$ preserves the same guarantee. Implement $\phi_{\theta}$ as an input-convex neural network by constraining hidden-to-hidden weights to be elementwise nonnegative and using convex nondecreasing activations, optionally adding a positive quadratic term to improve coercivity.

## Key formulas

- $\mathbf{g}=\mathcal{N}(\mathbf{H}\mathbf{x}^{\star}+b),\qquad \mathbf{H}_{i,j}\geq 0,\quad \mathbf{H}^{\top}\mathbf{1}=\mathbf{1},\quad b\geq 0.$
- $D_{\gamma}(x)=x-\gamma\nabla\phi_{\theta}(x),\qquad D_{\gamma,\rho}(x)=(1-\rho)x+\rho D_{\gamma}(x)=x-\rho\gamma\nabla\phi_{\theta}(x).$
- $\langle \nabla\phi_{\theta}(x)-\nabla\phi_{\theta}(y),x-y\rangle\geq \frac{1}{L}\left\|\nabla\phi_{\theta}(x)-\nabla\phi_{\theta}(y)\right\|^{2},\qquad 0<\rho\gamma\leq \frac{1}{L}.$
- $\left\|D_{\gamma,\rho}(x)-D_{\gamma,\rho}(y)\right\|^{2}\leq\left\langle D_{\gamma,\rho}(x)-D_{\gamma,\rho}(y),x-y\right\rangle.$

## Implementation notes

Integrate this as a replacement for a residual CNN denoiser in a repeated refinement network, plug-and-play ADMM loop, or diffusion-style restoration step. The tensor entering the block is an image or feature map $x$; flatten only for notation, while implementing $\phi_{\theta}$ with convolutional layers. Build an ICNN potential using nonnegative hidden-to-hidden convolution weights, convex nondecreasing activations such as ReLU or softplus, and an optional term $\frac{\mu}{2}\|x\|^{2}$ with $\mu\geq 0$. Compute the denoiser by automatic differentiation: `v = grad(phi_theta(x), x); x_new = x - rho*gamma*v`. Use this loop: `(1) x <- input; (2) phi <- ICNN(x); (3) v <- d phi / d x; (4) x <- x - rho*gamma*v; (5) repeat K times or feed x_new to the next reconstruction stage`. Train on clean/noisy image pairs with a denoising loss such as $\|D(x_{noisy})-x_{clean}\|_2^2$, plus a small gradient penalty to prevent exploding slopes. The mathematical guarantee is computed from convex architectural constraints and a bound $L$; estimate $L$ by power iteration on Jacobian-vector products, $L\approx\max_j\|J_{\nabla\phi}(u_j)\|_2$, over validation samples, or use a conservative product-of-layer-norm bound. Set $\rho\gamma=0.9/L$ and compare against an unconstrained DnCNN residual block and a spectrally normalized block. First test on $32\times32$ MNIST or CIFAR-10 images with Gaussian and Poisson corruption, using 5--10 repeated applications. Success means lower worst-case amplification $\|x_K-y_K\|/\|x_0-y_0\|$, no exploding iterates under stronger corruption than seen in training, and competitive PSNR at equal parameter count; for a PnP test, also measure reconstruction PSNR and sensitivity across a 10x range of ADMM penalties.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: inconclusive
- Verdict: Built and ran a reproducible quadratic ICNN-special-case verification plus a small learned denoising comparison. The predicted firm-nonexpansiveness boundary was confirmed at alpha*L=1.00 passing and 1.01 failing; the repeated-update divergence boundary was confirmed at alpha*L=2.00, with 2.01 diverging; and the K-step contraction sweep at alpha*L=0.9 exactly matched the spectral prediction. The learned convex-gradient denoiser also had lower test MSE and lower 10-step paired sensitivity than the unconstrained residual baseline, although this is only a small illustrative signal.

### Mechanism check

- Verdict: Built and ran a reproducible quadratic ICNN-special-case verification plus a small learned denoising comparison. The predicted firm-nonexpansiveness boundary was confirmed at alpha*L=1.00 passing and 1.01 failing; the repeated-update divergence boundary was confirmed at alpha*L=2.00, with 2.01 diverging; and the K-step contraction sweep at alpha*L=0.9 exactly matched the spectral prediction. The learned convex-gradient denoiser also had lower test MSE and lower 10-step paired sensitivity than the unconstrained residual baseline, although this is only a small illustrative signal.
- Confidence: 9/10
- Limitations: The learned model is a quadratic PSD-potential special case rather than a convolutional multi-layer ICNN. The baseline is an affine residual map rather than a full DnCNN or spectrally normalized CNN. No MNIST/CIFAR, Poisson, PnP-ADMM, distribution-shift, FLOP-matched, or multi-seed evaluation was performed.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 1
- Paired wins: 7/8
- Benchmark verdict: no significant win

## Artifacts

- [bench_report.json](https://synthcore.org/code/573/bench_report.json)
- [bench_stage2.py](https://synthcore.org/code/573/bench_stage2.py)
- [experiment.py](https://synthcore.org/code/573/experiment.py)
- [report.md](https://synthcore.org/code/573/report.md)
- [report_bench_2026-08-31T181715.md](https://synthcore.org/code/573/report_bench_2026-08-31T181715.md)
- [results.json](https://synthcore.org/code/573/results.json)
- [Download all files as ZIP](https://synthcore.org/download/573)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.