Reciprocal Feasibility-Preserving Optimizer

1. Integration point: modify the optimizer update applied to neural-network parameters \(\theta\), or to policy parameters in constrained RL. Define a batch of differentiable inequalities \(g_i(\theta)<0\); examples include \(\|\theta\|_2^2-B<0\), an estimated worst-case loss minus a safety threshold, action-limit violations, or a differentiable FLOP/memory budget. Start from a strictly feasible initialization. 2. Pseudocode: set \(k=k_0\), compute the task loss \(L(\theta)\), constraints \(g_i\), and gradients; set \(\lambda_i=-k/(g_i+\epsilon)\); form \(v=-\nabla_\theta L-\sum_i\lambda_i\nabla_\theta g_i\); choose an adaptive step \(h\), update \(\theta\leftarrow\theta+h v\), then recompute every constraint. Use \(k\leftarrow\max(k_{\min},k\exp(-h/\tau))\). Before accepting a step, cap \(h\) using \(h\le \eta\min_i[-g_i/(|\nabla g_i^\top v|+\epsilon)]\), with \(0<\eta<1\); reject and halve \(h\) if any constraint is nonnegative. 3. The reciprocal multiplier, barrier force, and annealing schedule come from the paper; the step cap and finite-step rejection are engineering safeguards because the guarantee concerns continuous time. Estimate constraint gradients with automatic differentiation and optionally clip \(\lambda_i\) to prevent floating-point overflow. 4. First experiment: train a small PPO policy on a constrained continuous-control task such as Safety Gymnasium, comparing Adam, projected-gradient updates, and this optimizer at equal environment interactions. Log task return, cumulative constraint violation, minimum constraint margin, rejected steps, and wall-clock cost. Success means zero or substantially lower intermediate violations with comparable final return, and fewer expensive safety QP calls than the projected baseline. A second cheap test can impose a differentiable parameter-norm budget during CIFAR-10 MLP training and measure accuracy at fixed budget and the fraction of steps crossing the boundary.