Decision-Dependent Robustness Cutting Planes
Implementation & benchmark of arXiv:2609.01538 — Generalized Semi-Infinite Programming for Robust Optimal Control with Decision-Dependent Uncertainty
Source paper: Generalized Semi-Infinite Programming for Robust Optimal Control with Decision-Dependent Uncertainty arXiv:2609.01538 ⓘ · analyzed Sep 2, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace ordinary adversarial training over a fixed perturbation set with adaptive robust training in which the admissible perturbations depend on the current network state. Train on a small active set of hard scenarios, then search for a newly admissible scenario with larger loss or constraint violation and add it only when needed. This should reduce redundant adversarial examples while targeting worst-case regions induced by the current model.
Formulas
Mathematical statement
The paper formulates a generalized semi-infinite program as $\min_{z\in\mathcal{Z}} f(z)$ subject to $g(z,v)\leq0$ for every $v\in\mathcal{V}(z)$, where $z$ is the finite-dimensional decision, $v$ is an index or uncertainty variable, $f$ is the objective, and $g$ is a robust constraint. The decision-dependent uncertainty set is $\mathcal{V}(z)=\{v\in\bar{\mathcal{V}}\mid h(z,v)\geq0,\ r(z,v)=0\}$, where $\bar{\mathcal{V}}$ is a bounded fixed superset, $h$ contains inequality feasibility functions, and $r$ contains equality feasibility functions. Adapt $z$ to neural-network parameters $\theta$, let $v$ be an input perturbation or environment scenario, and define $g(\theta,v)=\ell(\theta;v)-\rho$, where $\ell$ is scenario loss and $\rho$ is a target loss. The inner oracle searches for a feasible $v$ maximizing $g$. To make the oracle differentiable, use the penalty $\Phi_\tau(\theta,v)=g(\theta,v)-\lambda_h\tau\log\sum_i\exp(-h_i(\theta,v)/\tau)-\lambda_r\|r(\theta,v)\|_2^2$, with temperature $\tau>0$ and penalty weights $\lambda_h,\lambda_r>0$. For an active scenario set $A$, optimize the smooth robust surrogate $F_A(\theta)=f(\theta)+\beta\tau_A\log\sum_{v_k\in A}\exp(g(\theta,v_k)/\tau_A)$, where $\beta$ is robust-loss weight and $\tau_A$ controls approximation to the maximum. The key property is finite active-set optimization combined with separation: the oracle either finds a violating admissible scenario or certifies that the current discretization is approximately robust.
Implementation notes
Integrate the method at the training-loss level for a classifier or policy network. Let $\theta$ denote network parameters, $x$ a minibatch input, and $v$ encode a perturbation or environment scenario. Define $g(\theta,v)=\ell(f_\theta(x,v),y)-\rho$. Define $h(\theta,v)$ and $r(\theta,v)$ to represent model-dependent admissibility; for example, use a perturbation radius depending on predictive entropy, or require a representation-preservation equality $r(\theta,v)=\|\phi_\theta(x+v)-\phi_\theta(x)\|_2-c=0$. Maintain an active set $A$ initialized with the clean scenario and several random admissible scenarios. For each outer iteration, perform $T_A$ gradient updates on $F_A(\theta)$ using the displayed log-sum-exp formula. Then freeze $\theta$ and run an inner oracle from the worst active scenario plus random restarts: maximize $\Phi_\tau(\theta,v)$ over bounded $v\in\bar{\mathcal V}$ using projected Adam or projected gradient ascent. Gradually decrease $\tau$ and increase $\lambda_h,\lambda_r$; accept a candidate only if numerical feasibility satisfies $\min_i h_i(\theta,v)\geq-\epsilon_h$ and $\|r(\theta,v)\|_2\leq\epsilon_r$. If its loss exceeds the current active maximum by more than $\epsilon_g$, append it to $A$; otherwise end the cutting-plane round. Stop gradients through $\theta$ during oracle search. The formulas, active-set logic, and separation criterion come from the paper; the penalty relaxation and scenario-loss definitions are the neural-network adaptation. First test on CIFAR-10 with ResNet-18 against standard PGD adversarial training under the same total inner-gradient budget. Use a radius depending on predictive entropy, evaluate with an independent constrained multi-start oracle, and measure robust accuracy, clean accuracy, active-set size, oracle calls, and total FLOPs. A useful signal is lower worst-case loss with fewer retained scenarios or faster loss descent at equal attack-gradient cost; failure is unbounded active-set growth, poor feasibility, or no gain over PGD.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.