{
 "artifacts": null,
 "category": "optimization",
 "description": "Replace an unrolled constrained inner optimization in a meta-learning or hyperparameter-learning system with a KKT-based single-level layer. Instead of imposing primal-dual complementarity exactly from the first iteration, solve a sequence of relaxed problems with decreasing complementarity tolerances, making early optimization smoother and reducing failures caused by degenerate active-set geometry.",
 "formulas_latex": [
  "$$\\begin{array}{rl}S(x)=\\mathop{\\rm argmin}_{z}\u0026f_{0}(x,z)\\\\ \\mbox{subject to}\u0026f_{i}(x,z)\\leq 0,\\quad i=1,\\ldots,p.\\end{array}$$",
  "$$\\lambda_{1}-\\lambda_{2}-\\lambda_{3}=0,\\qquad -\\lambda_{1}-\\lambda_{2}+{\\bf 1}=0,\\qquad s\\succeq 0,\\quad\\lambda\\succeq 0,\\quad \\lambda^{T}s=0.$$",
  "$$\\lambda^{T}s\\leq\\varepsilon_k,\\qquad \\varepsilon_{k+1}=\\rho\\,\\varepsilon_k\\quad(0\u003c\\rho\u003c1).$$",
  "$$\\mathcal{L}_{\\varepsilon_k}(x,z,s,\\lambda)=F_0(x,z)+\\frac{\\beta_k}{2}(\\lambda^Ts)^2,\\qquad \\beta_k=\\beta_0/\\varepsilon_k.$$"
 ],
 "id": 2983,
 "implementation": "Integrate this as a constrained inner adaptation layer in a meta-learning or hyperparameter-optimization model. Use an outer parameter theta, inner variable z, and convex inner objective f_0(theta,z), with inequalities represented as f_i(theta,z)\u003c=0. First implement a quadratic inner problem, such as ridge regression with nonnegative weights or an L1-ball-constrained classifier, so KKT residuals can be checked exactly. At continuation stage k, create z, primal slacks s, and dual multipliers lambda. Enforce primal feasibility through the slack equations, enforce stationarity of the lower Lagrangian, and enforce s\u003e=0 and lambda\u003e=0. Do not initially impose exact lambda^T s=0. Either impose lambda^T s\u003c=epsilon_k with an inequality-capable solver or add the penalty beta_k(lambda^T s)^2 to the outer objective. Initialize epsilon_0 to approximately 1% of the initial primal-dual product magnitude, solve the relaxed problem, warm-start stage k+1 from the previous solution, set epsilon_{k+1}=0.2 epsilon_k, and stop when epsilon_k is below 1e-5 times the number of inequalities. Differentiate the validation loss through the final solve using implicit differentiation of the KKT residual Jacobian; use finite differences for an initial prototype. Log stationarity, primal violation, dual violation, complementarity gap, solve time, and outer gradients separately. The first cheap experiment should compare this method with 20- or 50-step projected-gradient unrolling on a small CIFAR-10 or Omniglot few-shot task with a convex nonnegative inner classifier. Measure validation accuracy, failed solves, wall-clock time, and sensitivity to initialization. Success means fewer failed inner solves and smoother outer loss, with equal or better accuracy at fewer inner iterations.",
 "math_summary": "The lower problem is S(x)=argmin_z f_0(x,z) subject to f_i(x,z)\u003c=0. For a convex lower problem satisfying a constraint qualification such as Slater's condition, primal feasibility, dual feasibility, stationarity, and complementary slackness characterize an inner optimum. In the paper's example, z is the lower solution, s=(s_1,s_2,s_3) are nonnegative primal slacks, and lambda=(lambda_1,lambda_2,lambda_3) are nonnegative dual multipliers. The stationarity equations are lambda_1-lambda_2-lambda_3=0 and -lambda_1-lambda_2+1=0. Exact complementarity is lambda^T s=0; because every component of lambda and s is nonnegative, this scalar equation is equivalent to componentwise lambda_i,j s_i,j=0. The transferable continuation replaces exact complementarity by lambda^T s\u003c=epsilon_k or penalizes its violation, with epsilon_k decreasing over stages. The outer variable x or hyperparameter theta enters f_0 and the constraints, while z, s, and lambda are solved jointly.",
 "math_tags": [
  "optimization",
  "convex-analysis",
  "numerical-analysis"
 ],
 "ml_areas": [
  "fine-tuning",
  "optimizer",
  "training",
  "loss"
 ],
 "paper": {
  "arxiv_id": "2609.00644",
  "arxiv_url": "https://arxiv.org/abs/2609.00644",
  "summary_what_math_gives_to_ml": "The paper contributes an executable way to convert a convex lower-level optimization problem into a single nonlinear program by combining conic canonicalization, KKT conditions, and complementarity-gap continuation. This is transferable to constrained meta-learning and hyperparameter optimization, where the inner problem can be a convex adaptation, calibration, pruning, or sparse-estimation layer and the outer objective evaluates validation performance. The useful asset is not merely bilevel differentiation, which is already common, but a solver-oriented treatment of inequality constraints that avoids enforcing degenerate complementarity exactly from the first iteration. A practical neural-network adaptation is to solve the inner constrained module through a sequence of relaxed KKT problems with decreasing complementarity tolerance, then differentiate through the resulting approximate solution.",
  "title": "Disciplined Bilevel Programming",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 4,
  "usefulness": 6
 },
 "solves": [
  "stability",
  "accuracy",
  "sample-efficiency"
 ],
 "title": "Gap-Continuation KKT Meta-Layer",
 "url": "https://synthcore.org/idea/2983/gap-continuation-kkt-meta-layer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}