# Active-Set Constraint Projection Layer

- ID: 848
- Canonical URL: https://synthcore.org/idea/848/active-set-constraint-projection-layer
- API JSON: https://synthcore.org/api/idea/848.json
- API Markdown: https://synthcore.org/api/idea/848.md
- Verification status: unverified
- Source: [arXiv:2607.13256](https://arxiv.org/abs/2607.13256)
- Category: architecture
- Solves: speedup, stability, accuracy
- ML areas: optimizer, mlp, loss, training
- Math tags: optimization, convex-analysis, numerical-analysis, linear-algebra
- Ratings: usefulness 6/10; difficulty 6/10; novelty 6/10

## Idea description

Replace generic projected-gradient iterations for equality-plus-bilateral constraints with an active-set semismooth Newton layer. Each iteration fixes currently active lower and upper bounds and solves one structured saddle-point system, potentially converging in a few iterations when only a small subset of constraints is active.

## Mathematical statement

The extracted algorithm identifies active facet sets \(\mathcal A_a^k\) and \(\mathcal A_b^k\) at iteration \(k\), then solves a saddle-point system with active constraint Jacobians \(\bm T_a^k\) and \(\bm T_b^k\), equality equations, and complementarity rows. For an ML projection layer, let \(z\) be the unconstrained network output, \(w\) the projected output, \(Q\succ0\) a metric, \(Aw=b\) equality constraints, and \(\ell\leq Cw\leq r\) bilateral bounds. The layer solves \(\min_w\frac12(w-z)^TQ(w-z)\) subject to these constraints. At a fixed active set, the KKT equations impose \(C_{\mathcal A_a}w=\ell_{\mathcal A_a}\) and \(C_{\mathcal A_b}w=r_{\mathcal A_b}\). Multipliers with incorrect signs are released, while violated inactive constraints are added. The paper's stopping principle is the primal-dual gap, not merely the residual of the linear system.

## Key formulas

- $$\min_{w}\;\frac12(w-z)^TQ(w-z)\quad\text{subject to}\quad Aw=b,\qquad \ell\leq Cw\leq r$$
- $$\begin{bmatrix}Q&A^T&C_{\mathcal A_a}^T&C_{\mathcal A_b}^T\\A&0&0&0\\C_{\mathcal A_a}&0&0&0\\C_{\mathcal A_b}&0&0&0\end{bmatrix}\begin{bmatrix}w\\y\\\lambda_a\\\lambda_b\end{bmatrix}=\begin{bmatrix}Qz\\b\\\ell_{\mathcal A_a}\\r_{\mathcal A_b}\end{bmatrix}$$
- $$\mathcal A_a^{k+1}=\{i:(Cw^k)_i-\ell_i\leq\tau\},\qquad \mathcal A_b^{k+1}=\{i:r_i-(Cw^k)_i\leq\tau\}$$
- $$\eta_{\mathrm{gap}}^2=P(w)-D(y,\lambda)\leq\varepsilon_{\mathrm{STOP}}$$

## Implementation notes

Insert the layer immediately after the final linear output and before the task loss. It receives unconstrained z, fixed A, C, lower bounds ell, upper bounds r, and a tolerance tau. Initialize w=z and identify lower-active indices where Cw-ell<=tau and upper-active indices where r-Cw<=tau. Assemble and solve the displayed KKT system using the current active rows. Recompute w and multipliers, release a lower constraint if its multiplier has the wrong sign, release an upper constraint if its multiplier has the wrong sign, and add any violated inactive constraint. Repeat for at most 5-10 iterations. Pseudocode is: active_a,active_b=detect(w); for k in range(K): solve KKT(Q,A,C[active_a],C[active_b],z,b,ell,r); remove lower indices with lambda_a<0 and upper indices with lambda_b<0; add inactive indices with Cw<ell-tau or Cw>r+tau; stop if maximum violation<tau and gap<epsilon. Backpropagate through the fixed-active-set linear solve using implicit differentiation; use straight-through active-set decisions when an active set changes. The paper supplies the active-set KKT structure and gap-based stopping rule. Use a sparse factorization when A and C are sparse, and damp updates if active sets oscillate. First test a 100-dimensional allocation task with one sum constraint and box bounds. Compare against clipping plus renormalization and 20-step projected gradient descent. Measure forward latency, exact feasibility, number of linear solves, gradient norms near bound transitions, and task loss. Success means lower latency and constraint violation than projected gradient at equal accuracy.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.