Unverified 2026

Lifting-free PSD certificate layer

Implementation & benchmark of arXiv:2607.13701 — Lifting-Free Quadratic Sum-Of-Squares Programming

Usefulness6/10
Difficulty6/10
Novelty7/10

Source paper: Lifting-Free Quadratic Sum-Of-Squares Programming arXiv:2607.13701 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use the paper's norm-regularized conic dualization to impose PSD or SOS-style certificate constraints during neural-network training without forming Schur-complement or second-order-cone liftings. A neural dynamics model can be trained jointly with a polynomial Lyapunov or energy certificate, while the certificate subproblem is solved through accelerated updates in equality-constraint dual variables.

Formulas

$$\min_{z,s}\;\frac{1}{2}z^{\top}Qz+w^{\top}z+\frac{\mu}{2}\|s\|_F^2\quad\text{s.t.}\quad Az+Bs=b,\quad s\in K,$$
$$g_{\mu}(y)=\frac{1}{2}(w+A^{\top}y)^{\top}Q^{-1}(w+A^{\top}y)+y^{\top}b+\frac{1}{2\mu}\left\|\Pi_K(-B^{\top}y)\right\|_F^2,$$
$$z(y)=-Q^{-1}(w+A^{\top}y),\qquad s(y)=\frac{1}{\mu}\Pi_K(-B^{\top}y),\qquad \nabla g_{\mu}(y)=b+Az(y)+Bs(y),$$
$$y_{k+1}=y_k+\frac{1}{L}\nabla g_{\mu}(y_k),\qquad t_{k+1}=\frac{1+\sqrt{1+4t_k^2}}{2},\qquad y^{\mathrm{acc}}_{k+1}=y_{k+1}+\frac{t_k-1}{t_{k+1}}(y_{k+1}-y_k).$$

Mathematical statement

Represent a generic certificate subproblem as min_{z,s} (1/2)z^TQz+w^Tz subject to Az+Bs=b and s in K, where z contains free polynomial or neural coefficients, s is the SOS/PSD cone variable, K is a closed convex cone, Q is positive definite, and A,B,b encode coefficient-matching constraints. Add the lifting-free quadratic regularizer mu/2||s||_F^2, with mu>0. For a dual multiplier y, minimizing over z and s gives the unconstrained dual objective g_mu(y)=1/2(w+A^Ty)^TQ^{-1}(w+A^Ty)+y^Tb+(1/(2mu))||Pi_K(-B^Ty)||_F^2, up to the sign convention used for the Lagrangian. Here Pi_K is Euclidean projection onto K; for the PSD cone, Pi_K(X)=U diag(max(lambda_i(X),0)) U^T. Its gradient is the primal equality residual grad g_mu(y)=b+Az(y)+Bs(y), with z(y)=-Q^{-1}(w+A^Ty) and s(y)=Pi_K(-B^Ty)/mu. Since projection onto a closed convex cone is nonexpansive, the dual gradient is Lipschitz; a conservative bound is L <= ||AQ^{-1}A^T||_2+||B||_2^2/mu. The regularized solution approximates the original conic solution, with smaller mu reducing bias but worsening dual conditioning.

Implementation notes

Integrate this as a training-time certificate solver for a neural dynamics or energy model. Let the network f_theta(x) predict dynamics, and let V_c(x) be a polynomial Lyapunov candidate with coefficient vector z; optionally let a second network produce coefficients entering the equality constraints. Sample collocation points x_i and construct coefficient or value-matching equations Az+Bs=b for conditions such as V_c(x_i)>=epsilon||x_i||^2 and -nabla V_c(x_i)^T f_theta(x_i)>=epsilon||x_i||^2. Use a PSD Gram matrix for each SOS constraint, but do not introduce Schur-complement lifting variables. For each minibatch, run K dual iterations: initialize y=0; compute z=-Q^{-1}(w+A^T y); compute X=-B^T y; eigendecompose each symmetric X and set Pi_K(X)=U max(diag(lambda),0) U^T; set s=Pi_K(X)/mu; compute r=b+Az+Bs; update y with accelerated ascent using the displayed t recurrence and step size 1/L. Backpropagate through the unrolled iterations, or stop-gradient through the solver and alternate certificate and network updates. Estimate L with 5-10 power iterations on AQ^{-1}A^T+BB^T/mu, and reduce mu geometrically only after ||r|| falls below a threshold. The cheap first experiment is a 2-layer MLP learning a stable 2D or 4D nonlinear system, comparing this solver against a CVXPY SDP baseline and a penalty-only certificate loss. Measure peak memory, wall-clock time per certificate update, equality residual, minimum sampled Lyapunov margin, and closed-loop rollout stability. Success means at least 2x lower peak memory or a materially larger certificate degree at equal memory, while preserving the baseline certificate margin and reducing the constraint residual below 1e-4.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.