# ISS-CLF/RCBF Neural Policy Shield

- ID: 1532
- Canonical URL: https://synthcore.org/idea/1532/iss-clf-rcbf-neural-policy-shield
- API JSON: https://synthcore.org/api/idea/1532.json
- API Markdown: https://synthcore.org/api/idea/1532.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.28353](https://arxiv.org/abs/2607.28353)
- Category: dynamics
- Solves: stability, accuracy, sample-efficiency
- ML areas: rl, rnn, ssm, optimizer, training-dynamics
- Math tags: control-theory, dynamical-systems, optimization, geometry
- Ratings: usefulness 7/10; difficulty 5/10; novelty 5/10

## Idea description

Attach a small robust quadratic-program layer to a neural controller. The network proposes an action, and the QP returns the closest action satisfying an ISS Lyapunov decrease constraint and a robust safety-barrier constraint under bounded model disturbances. This should preserve the network's behavior away from constraint boundaries while preventing unstable or unsafe actions near those boundaries.

## Mathematical statement

For a continuous-time input-affine plant or learned latent dynamics, let x in R^n be the state, u in R^m the action, w in W an unknown disturbance, and x_dot = f_0(x) + g(x)u + p(x)w. Let V(x) >= 0 be an ISS control Lyapunov function and h(x) >= 0 define the safe set. Assume W = {w: ||w||_2 <= w_bar}. The worst-case disturbance contribution to the derivative of a differentiable scalar q is bounded by w_bar ||p(x)^T grad q(x)||_2. Enforce the ISS-CLF condition grad V(x)^T(f_0(x)+g(x)u) + w_bar ||p(x)^T grad V(x)||_2 + c_V V(x) <= s_V and the robust CBF condition grad h(x)^T(f_0(x)+g(x)u) - w_bar ||p(x)^T grad h(x)||_2 + alpha_h(h(x)) >= -s_h, where c_V > 0, alpha_h is an extended class-K function such as alpha_h(h)=c_h h, and s_V,s_h >= 0 are penalized slacks. With zero slacks, the Lyapunov condition predicts V(t) <= exp(-c_V t)V(0) + d_V(1-exp(-c_V t))/c_V when residual disturbance effects are bounded by d_V. The QP chooses the smallest deviation from the neural action while satisfying these inequalities and actuator bounds.

## Key formulas

- $$\dot{x}=f_0(x)+g(x)u+p(x)w,\qquad \mathcal{W}=\{w\in\mathbb{R}^{n_w}:\|w\|_2\le \bar w\}.$$
- $$\nabla V(x)^T\bigl(f_0(x)+g(x)u\bigr)+\bar w\left\|p(x)^T\nabla V(x)\right\|_2+c_VV(x)\le s_V.$$
- $$\nabla h(x)^T\bigl(f_0(x)+g(x)u\bigr)-\bar w\left\|p(x)^T\nabla h(x)\right\|_2+\alpha_h(h(x))\ge -s_h.$$
- $$u^*(x)=\arg\min_{u,s_V,s_h}\frac12\|u-u_\theta(x)\|_{R}^{2}+\rho_Vs_V^2+\rho_hs_h^2\quad\text{s.t. the CLF, CBF, }u\in\mathcal U,\ s_V,s_h\ge0.$$

## Implementation notes

1. Integration point: use a neural policy u_theta(x) in an RL agent, an RNN/SSM controller, or a learned latent-dynamics controller. At every control step, evaluate the policy once and pass its action through a tiny QP layer. For an initial MVP, use a known two-dimensional plant such as a pendulum or point mass, choose V(x)=||x-x_goal||_2^2 and h(x) as the signed distance to obstacles or state limits, and use box constraints for u. 2. Pseudocode: compute f_0(x), g(x), and p(x); compute u_net=u_theta(x); compute gradients of V and h by autodiff; calculate r_V=w_bar ||p^T grad V||_2 and r_h=w_bar ||p^T grad h||_2; solve the displayed QP; execute u*. During training, backpropagate through a differentiable QP solver or stop gradients through the projection initially. 3. Computed quantities are the CLF/CBF gradients, nominal vector fields, actuator bounds, and analytic worst-case norm margins. Estimate w_bar from held-out model residuals using a high quantile, and log both slack variables and QP infeasibility. 4. First cheap experiment: train the same PPO or SAC MLP on a two-dimensional obstacle-avoidance task with and without the shield, using identical seeds and injected bounded disturbances. The quantitative prediction is that trajectories satisfying the CLF with negligible slack obey an empirical envelope V(t) approximately less than or equal to exp(-c_V t)V(0)+d_V/c_V; fit the decay rate and require it to agree with c_V within 20 percent. Increase w_bar until the robust margin crosses the available control authority; QP infeasibility and safety violations should exhibit a sharp transition near that predicted feasibility boundary. Compare action deviation, violation rate, and episode return.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built a scalar robust ISS-CLF/CBF policy shield with box-bounded control, analytic disturbance margins, slacked QP projection, and deterministic sweeps. The mechanism manifested: robust CLF margin scaled linearly with disturbance at observed slope 0.600 versus predicted 0.600, zero-slack feasibility ended in the predicted interval [0.80, 0.85] around the analytic boundary 0.80, and the adversarial trajectory stayed within the exponential Lyapunov envelope with max ratio 0.9999998 and residual 1.22e-6. The shield reduced safety violation fraction from 0.917 to 0 while requiring substantial action deviation, so this is evidence for safety behavior but not general RL performance.

### Mechanism check

- Verdict: Built a scalar robust ISS-CLF/CBF policy shield with box-bounded control, analytic disturbance margins, slacked QP projection, and deterministic sweeps. The mechanism manifested: robust CLF margin scaled linearly with disturbance at observed slope 0.600 versus predicted 0.600, zero-slack feasibility ended in the predicted interval [0.80, 0.85] around the analytic boundary 0.80, and the adversarial trajectory stayed within the exponential Lyapunov envelope with max ratio 0.9999998 and residual 1.22e-6. The shield reduced safety violation fraction from 0.917 to 0 while requiring substantial action deviation, so this is evidence for safety behavior but not general RL performance.
- Confidence: 8/10
- Limitations: Only a scalar known plant was tested; no neural policy training, differentiable QP backpropagation, learned dynamics, multidimensional obstacles, stochastic seeds, runtime profiling, or PPO/SAC comparison was performed. The QP uses active-set enumeration rather than a production solver, and the feasibility sweep resolution is 0.05.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/596/bench_report.json)
- [bench_shield.py](https://synthcore.org/code/596/bench_shield.py)
- [math_sanity.py](https://synthcore.org/code/596/math_sanity.py)
- [report.md](https://synthcore.org/code/596/report.md)
- [report_bench_2026-08-31T193518.md](https://synthcore.org/code/596/report_bench_2026-08-31T193518.md)
- [results.json](https://synthcore.org/code/596/results.json)
- [shield_experiment.py](https://synthcore.org/code/596/shield_experiment.py)
- [Download all files as ZIP](https://synthcore.org/download/596)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.