Perspective Proximal Fine-Tuning Solver

Integrate this as an inner optimizer for a convex parameter block, rather than replacing backpropagation through all nonlinear network weights. A minimal target is a frozen transformer with a trainable linear classification head, linear probe, or low-dimensional convex adapter. Let w be the trainable vector, let f(w)=(1/N)\sum_i \ell(y_i,\phi_i^ op w) be logistic or squared data-fit loss, and let g(w) be an L2, group-L1, or box-constraint indicator. At each outer epoch, form the fixed feature matrix phi and run a perspective Douglas-Rachford inner solve. Pseudocode: initialize the splitting state z and p=1; repeat u=prox_{gamma g}(z); set v=2u-z; compute (x_tilde,p_tilde) by projecting v onto the perspective embedding; solve the projection's scalar p equation with safeguarded Newton or bisection; evaluate prox_{gamma f} for the parameter component; update z <- z + (x_tilde,p_tilde)-u; stop on primal and dual residuals; return w=x_tilde/p_tilde if p_tilde>epsilon. For squared loss, implement the proximal step with a linear solve; for logistic loss, use 3-5 Newton steps or a one-dimensional separable prox approximation. Estimate the scalar root numerically, while computing residuals and certificates directly from the embedding. Track p, primal residual, dual residual, objective gap, and whether p approaches zero with a nonzero normalized certificate. First experiment: train a linear head on frozen MNIST or CIFAR-10 features with box constraints, comparing projected gradient, projected Adam, and ordinary Douglas-Rachford at equal matrix-vector FLOPs. Success means fewer iterations to a fixed validation loss, lower sensitivity when multiplying the loss by 10 or 100, and correct detection of deliberately contradictory constraints. A negative result is no improvement over ordinary proximal splitting or frequent collapse of p toward zero on feasible problems.