Unverified 2026

Condensed primal-dual training for constrained neural dynamics

Implementation & benchmark of arXiv:2608.15582 — Condensed PIPG Sequential Convex Optimization for Reusable-Rocket Powered Landing with Strong Aerodynamics

Usefulness6/10

Difficulty7/10

Novelty7/10

Optimization Speedup Stability Memory Optimizer Ssm World model Training dynamics Optimization Linear algebra Dynamical systems Numerical analysis

Source paper: Condensed PIPG Sequential Convex Optimization for Reusable-Rocket Powered Landing with Strong Aerodynamics arXiv:2608.15582 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace a generic optimizer over every discretized hidden state in a neural ODE or state-space model with a condensed reduced-space solve. At each outer Gauss-Newton or sequential-convex-programming iteration, linearize the neural dynamics, recursively eliminate all intermediate state increments, and apply projected primal-dual gradient updates to the remaining model parameters, controls, and terminal variables. This should be most useful when a model is trained with hard terminal targets, conservation laws, or collocation defects.

Formulas

$$e_{\mathrm{dyn}}=\max_{k,i}\left|\frac{x_{k+1,i}-x_{k,i}-\dfrac{ht_f}{2}(f_{k,i}+f_{k+1,i})}{s_{x,i}}\right|$$

$$\delta x_{k+1}-J^x_k\delta x_k-J^\theta_k\delta\theta-J^u_k\delta u_k=r_k$$

$$\delta x_k=\Phi_{k,0}\delta x_0+\sum_{j=0}^{k-1}\Phi_{k,j+1}\left(J^\theta_j\delta\theta+J^u_j\delta u_j+r_j\right),\qquad \Phi_{k,j}=J^x_{k-1}\cdots J^x_j$$

$$z^{j+1}=\Pi_{\mathcal C}\!\left[z^j-\tau(Qz^j+c+A^{\mathsf T}\lambda^j)\right],\qquad \lambda^{j+1}=\lambda^j+\sigma\left(A(2z^{j+1}-z^j)-b\right)$$

Mathematical statement

The paper validates discretized dynamics using the trapezoidal defect formula, where x_k is the state at node k, h is normalized node spacing, t_f is final time, and f_k is the physical dynamics. Its central algorithmic structure eliminates state variables and dynamics equalities before solving the convex subproblem. For a neural dynamical system x_{k+1}=F_k(x_k,theta,u_k), linearize around the current trajectory: delta x_{k+1}-J^x_k delta x_k-J^theta_k delta theta-J^u_k delta u_k=r_k, where J^x_k=partial F_k/partial x_k, J^theta_k=partial F_k/partial theta, J^u_k=partial F_k/partial u_k, and r_k is the current defect. Define Phi_{k,j}=J^x_{k-1}J^x_{k-2}...J^x_j for k>j. Forward elimination gives delta x_k=Phi_{k,0} delta x_0+sum_{j=0}^{k-1} Phi_{k,j+1}(J^theta_j delta theta+J^u_j delta u_j+r_j), so all intermediate delta x_k are affine functions of the reduced variables (delta x_0,delta theta,delta u). The terminal equality C_f x_N=x_f becomes a small reduced equality A z=b, where z is the reduced variable vector. For a quadratic local model min_z 1/2 z^T Q z+c^T z subject to A z=b and z in C, use the projected primal-dual iteration with primal step tau>0, dual step sigma>0, and projection set C. The paper's terminal residual and nonlinear defect checks adapt directly to e_f=||S_f(C_f x_N-x_f)||_infinity and the scaled nonlinear defect formula.

Implementation notes

(1) Integration point: use this as the inner optimizer for a discretized neural ODE, recurrent world model, or structured SSM trained with hard terminal supervision and optional bounded controls. Keep the normal forward simulation for generating the reference trajectory, but do not expose every hidden state increment as an independent optimization variable in the inner quadratic subproblem. The reduced variables should be the initial-state correction, trainable parameter correction, control corrections, and any small set of slack variables. (2) Pseudocode: initialize a reference rollout x_0,...,x_N; compute J^x_k,J^theta_k,J^u_k with automatic differentiation and defects r_k; form or apply the condensed affine map by the recursion delta x_{k+1}=J^x_k delta x_k+J^theta_k delta theta+J^u_k delta u_k+r_k; build the reduced Gauss-Newton quadratic (Q,c) and terminal matrix (A,b); set z and lambda to zero; for j=1,...,J, perform the projected primal update and dual update above; recover every delta x_k by the same forward recursion; line-search or trust-region accept the step only if the original nonlinear rollout reduces e_f+e_dyn. (3) Compute Jacobians, condensed products, and the quadratic model from the linearization; estimate tau and sigma by a few power iterations estimating ||A||_2 and ||Q||_2, using conservative conditions tau sigma ||A||_2^2<0.9 and tau||Q||_2<1. The projection Pi_C should enforce parameter trust regions, control bounds, and positive slack variables. Recompute nonlinear defects with the original neural dynamics rather than trusting the linearized constraints. (4) First experiment: train a 2-layer neural ODE or GRU-like SSM on a synthetic damped oscillator and a small sequential-MNIST task, comparing Adam plus penalty losses, a full-state SQP or least-squares solver, and the condensed primal-dual method at equal Jacobian evaluations. Use 32--64 nodes and impose a terminal-state target. Measure wall-clock time per outer iteration, peak memory, terminal error, nonlinear defect, and accepted-step fraction. A successful result is at least 2x lower inner-solve latency or 30% lower peak memory at matched terminal accuracy, with fewer failed outer steps and no growth in e_dyn.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.