# Residual-Gated DRS Solver Layer

- ID: 2293
- Canonical URL: https://synthcore.org/idea/2293/residual-gated-drs-solver-layer
- API JSON: https://synthcore.org/api/idea/2293.json
- API Markdown: https://synthcore.org/api/idea/2293.md
- Verification status: mechanism_failed
- Source: [arXiv:2608.14470](https://arxiv.org/abs/2608.14470)
- Category: optimization
- Solves: speedup, stability, accuracy
- ML areas: optimizer, training, architecture
- Math tags: optimization, convex-analysis, dynamical-systems, control-theory
- Ratings: usefulness 7/10; difficulty 5/10; novelty 6/10

## Idea description

Replace fixed-parameter unrolled Douglas–Rachford iterations in a differentiable convex optimization layer with a causal controller that adapts relaxation and objective-drive strength from the current residuals. The controller should accelerate early progress while enforcing admissible parameter ranges, so every individual block remains a stable relaxed splitting map rather than an unconstrained learned optimizer.

## Mathematical statement

The conic linear program is $$\min_x c^\top x$$ subject to $$Ax=b$$ and $$x\in\mathcal K,$$ where $$A\in\mathbb R^{m\times n}$$, $$b\in\mathbb R^m$$, $$c\in\mathbb R^n$$, and $$\mathcal K$$ is a Cartesian product of closed convex cones with computable projections. Its affine projection is the paper's explicit formula $$\operatorname{Proj}_{\mathcal A}(v)=v-A^\top(AA^\top)^\dagger(Av-b),$$ with $$\mathcal A=\{x:Ax=b\}$$. For the splitting state $$z_t$$, define $$u_t=P_{\mathcal K}(z_t)$$ and $$v_t=\operatorname{prox}_{\gamma f}(2u_t-z_t)=\operatorname{Proj}_{\mathcal A}(2u_t-z_t-\gamma\beta_t c),$$ where $$\gamma>0$$ is the step size and $$\beta_t\ge0$$ controls objective drive. The relaxed DRS update is $$z_{t+1}=z_t+\lambda_t(v_t-u_t)$$, with relaxation $$0<\lambda_t<2$$. The unrelaxed DRS operator is firmly nonexpansive, so the relaxed operator is averaged with parameter $$\alpha_t=\lambda_t/2$$. For a fixed $$\alpha$$-averaged operator $$T$$ and fixed point $$z^\star$$, the paper gives the finite-step residual bound $$\min_{0\le t<N}\|z_t-Tz_t\|^2\le\frac{\alpha}{1-\alpha}\frac{\|z_0-z^\star\|^2}{N}$$. The controller uses the observable fixed-point residual $$r_t=\|v_t-u_t\|_2$$ and affine feasibility residual $$p_t=\|Av_t-b\|_2$$. Cone feasibility is checked by $$k_t=\|u_t-P_{\mathcal K}(u_t)\|_2$$, which should be near projection numerical error.

## Key formulas

- $$\operatorname{Proj}_{\mathcal A}(v)=v-A^{\top}(AA^{\top})^{\dagger}(Av-b),\qquad \mathcal A=\{x\in\mathbb R^n:Ax=b\}.$$
- $$u_t=P_{\mathcal K}(z_t),\qquad v_t=\operatorname{Proj}_{\mathcal A}\!\left(2u_t-z_t-\gamma\beta_t c\right).$$
- $$z_{t+1}=z_t+\lambda_t(v_t-u_t),\qquad 0<\lambda_t<2,$$
- $$\min_{0\le t<N}\|z_t-Tz_t\|^2\le \frac{\alpha}{1-\alpha}\frac{\|z_0-z^\star\|^2}{N},\qquad \alpha=\frac{\lambda}{2}.$$

## Implementation notes

(1) Integration point: use this as a replacement for a fixed-depth differentiable optimization layer inside a predictor-to-optimizer network. The network emits or predicts problem data that are assembled into $$A,b,c$$; the layer returns $$x_T=u_T$$ and optionally residual diagnostics. Keep the cone projection and affine projection differentiable through autodiff or custom backward rules. Fix $$\gamma$$ initially and adapt only $$\lambda_t$$ and $$\beta_t$$. (2) Pseudocode: initialize $$z\leftarrow z_0$$, $$\lambda\leftarrow1$$, $$\beta\leftarrow1$$, and $$r_{-1}=\infty$$. At each iteration compute $$u=P_K(z)$$, then $$v=P_A(2u-z-\gamma\beta c)$$ using $$P_A(w)=w-A^\top(AA^\top)^\dagger(Aw-b)$$. Compute $$r=\|v-u\|_2$$, $$p=\|Av-b\|_2$$, and objective progress $$q=|c^\top u-c^\top u_{\mathrm{prev}}|$$. Update relaxation with $$\lambda\leftarrow\operatorname{clip}(\lambda\exp(\eta_\lambda\operatorname{clip}(\log((r_{\mathrm{prev}}+\epsilon)/(r+\epsilon)),-1,1)),\lambda_{\min},\lambda_{\max})$$, where $$0<\lambda_{\min}<\lambda_{\max}<2$$. If $$r$$ decreases and $$p$$ does not worsen, increase $$\beta$$ mildly toward $$\beta_{\max}$$; if $$r$$ increases by more than 10 percent or $$p$$ worsens, decrease $$\beta$$ and set $$\lambda\leftarrow\max(\lambda_{\min},0.8\lambda)$$. Then apply $$z\leftarrow z+\lambda(v-u)$$. To approximate the paper's safeguarded time-varying rollout, multiply controller step sizes by $$\rho^t$$ with $$0<\rho<1$$, or freeze $$\lambda,\beta$$ after a warmup block, making total parameter variation finite. (3) Computed versus estimated: affine and cone projections are computed exactly or to numerical tolerance; $$r,p,q$$ and all controller updates are empirical online quantities. The averaged-operator guarantee applies whenever $$\lambda$$ remains in its admissible interval; the controller does not need to know $$z^\star$$. (4) First experiment: implement a 10- to 20-step layer for small feasible quadratic programs and second-order-cone programs. Compare fixed $$\lambda=1,\beta=1$$, grid-tuned fixed parameters, and the controller at equal projection calls and FLOPs. Train a two-layer MLP that predicts problem coefficients on randomly generated feasible instances. Measure objective suboptimality, equality and cone violation, fixed-point residual, and downstream task loss versus depth. The expected signal is lower residual and feasibility error at the same depth, or the same target optimization accuracy with 25–50 percent fewer unrolled iterations; instability should not increase because $$\lambda$$ is clipped below 2.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a NumPy residual-gated DRS solver with affine projection, clipped adaptive lambda/beta control, exact toy sweeps, and a simplex-LP comparison. The mathematical checks succeeded: affine projection errors were about 1e-15, the exact residual rate matched |1-lambda| for every tested lambda, and contraction changed exactly at lambda=2; the finite-step residual bound also held with ratios below 3e-6. However, on 100 matched 24-step simplex LPs the controller was worse: median residual 0.1030 versus 0.0245, median equality violation 0.1292 versus 0.0229, and residual wins on only 2% of trials, so the proposed controller did not demonstrate a practical win.

### Mechanism check

- Verdict: Built a NumPy residual-gated DRS solver with affine projection, clipped adaptive lambda/beta control, exact toy sweeps, and a simplex-LP comparison. The mathematical checks succeeded: affine projection errors were about 1e-15, the exact residual rate matched |1-lambda| for every tested lambda, and contraction changed exactly at lambda=2; the finite-step residual bound also held with ratios below 3e-6. However, on 100 matched 24-step simplex LPs the controller was worse: median residual 0.1030 versus 0.0245, median equality violation 0.1292 versus 0.0229, and residual wins on only 2% of trials, so the proposed controller did not demonstrate a practical win.
- Confidence: 9/10
- Limitations: Only small NumPy toy/simplex LP experiments were tested; no PyTorch autodiff integration, learned predictor-to-optimizer network, second-order cones, FLOP/runtime profiling, grid-tuned baseline, or training comparison was performed.

## Artifacts

- [experiment.py](https://synthcore.org/code/912/experiment.py)
- [report.md](https://synthcore.org/code/912/report.md)
- [results.json](https://synthcore.org/code/912/results.json)
- [Download all files as ZIP](https://synthcore.org/download/912)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.