✗ Failed on benchmark 2026

Adaptive Zonotope Safety Shield

Implementation & benchmark of arXiv:2608.26852 — Towards Safe Reinforcement Learning with Reduced Conservativeness: A Case Study on Drone Flight Control

Usefulness7/10

Difficulty6/10

Novelty7/10

Dynamics Stability Accuracy Sample efficiency Rl Optimizer Training dynamics Control theory Dynamical systems Linear algebra Geometry

Source paper: Towards Safe Reinforcement Learning with Reduced Conservativeness: A Case Study on Drone Flight Control arXiv:2608.26852 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Wrap a neural policy with an online disturbance estimator and a zonotopic reachability shield. Instead of rejecting actions using a permanently worst-case disturbance set, update the disturbance zonotope from observed transition residuals and accept an action only when the resulting reachable set remains inside the safe region.

Formulas

x_(t+1) = f(x_t, u_t, w_t), X_(t+1) = A_t X_t plus B_t u_t plus d_t plus W_t

X_t = c_x,t + G_x,t [-1,1]^p, W_t = c_w,t + G_w,t [-1,1]^q

c_x,(t+1) = A_t c_x,t + B_t u_t + d_t + c_w,t, G_x,(t+1) = [A_t G_x,t, G_w,t]

u_t is accepted if and only if X_(t+1)(u_t) is a subset of S; q_j <- max((1-beta)q_j, max_(i in batch) |r_i,j - c_w,j| + epsilon)

Mathematical statement

For a discrete-time plant x_(t+1) = f(x_t, u_t, w_t), use a local affine model x_(t+1) approximately equals A_t x_t + B_t u_t + d_t + w_t, where x_t is the state, u_t is the neural-policy action, A_t and B_t are local Jacobians or fitted linear dynamics, d_t is a nominal offset, and w_t is bounded disturbance. Represent the current state and disturbance as zonotopes X_t = c_x,t + G_x,t [-1,1]^p and W_t = c_w,t + G_w,t [-1,1]^q. The one-step reachable set is X_(t+1) = A_t X_t plus B_t u_t plus d_t plus W_t, with center c_x,(t+1) = A_t c_x,t + B_t u_t + d_t + c_w,t and generator matrix G_x,(t+1) = [A_t G_x,t, G_w,t]. Given a safety set S, accept the action only if an over-approximation of X_(t+1) is contained in S. For observed residuals r_i = x_(i+1) - f_nom(x_i,u_i), update the disturbance center and axis-aligned radii using c_w,j <- (1-alpha)c_w,j + alpha median_i(r_i,j) and q_j <- max((1-beta)q_j, max_i |r_i,j - c_w,j| + epsilon), where q_j defines W_t = c_w + diag(q) zeta with ||zeta||_infinity <= 1, alpha and beta are adaptation rates, and epsilon is a robustness margin.

Implementation notes

1. Integration point: place the shield between a policy network pi_theta and the environment, and optionally use the same reachable-set violation as a policy-training penalty. For a proposed action, linearize a differentiable simulator around the current state, or estimate A_t and B_t with finite differences or a small local dynamics model. 2. Pseudocode: propose u = policy(x); after the next state is observed, compute residual r = x_next - f_nom(x,u); update c_w and q using the robust batch rule above; construct X_next from the current state zonotope and W; if X_next is outside S, project u toward a backup action u_safe or replace it with the action minimizing zonotope constraint violation. During training, add lambda times normalized reachable-set violation to the policy loss. 3. Computed from the paper mechanism: zonotope propagation, set containment, and online disturbance refinement. Estimated empirically: local Jacobians, residual statistics, epsilon, and the probability that the estimated disturbance set covers future residuals. Maintain a held-out residual-coverage monitor and never shrink q below a calibrated quantile plus epsilon. 4. First cheap experiment: train PPO on a two-dimensional point mass with unknown bounded acceleration disturbances, comparing PPO, a fixed worst-case box shield, and the adaptive shield. Sweep disturbance magnitude and adaptation rate. The quantitative prediction is that the adaptive shield's reachable-set generator norm and action-rejection rate decrease after residual burn-in while empirical disturbance coverage remains above 1 minus delta. If the true disturbance scale jumps beyond the learned set, coverage should fail within one batch and q should expand. Measure the largest disturbance magnitude with zero constraint violations over 1000 episodes; the adaptive method should approach the fixed-shield safety boundary within 20 percent while rejecting fewer actions.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: No measurable effect

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built and numerically verified an adaptive axis-aligned disturbance zonotope shield with affine reachability and online median/max-radius updates. The mechanism manifested: zonotope box containment had 0 disagreements over 3000 cases, learned q tracked disturbance amplitude plus epsilon within 0.00292, and a disturbance jump expanded q in the first batch with 100% coverage. However, the tiny shield comparison showed no efficiency or safety win: adaptive rejection was 70.08% versus fixed 66.86%, with violation rates 67.49% versus 66.86%.

Agent confidence: 8/10

Baseline: Fixed disturbance box: rejection_rate=0.6686, violation_rate=0.6686
Idea: Adaptive disturbance box: rejection_rate=0.7008, violation_rate=0.6749; scaling q maximum absolute error=0.0029165, minimum held-out coverage=1.0; jump q=0.10362 to 0.31315 in one batch

Limitations: This is a deterministic toy affine point-mass experiment, not PPO or a learned nonlinear dynamics model. It uses axis-aligned disturbance generators, a zero-action backup, short finite episodes, and does not test multi-step reachability, calibration under distribution shift beyond one jump, computational overhead, or statistically repeated seeds.

How to run: python3 adaptive_shield_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.26852) Sep 2, 2026

Ahead of the baseline (-0.0001 metric points) but not statistically significant (p=0.5844, wins 4 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0
Idea mean: 0.0027
Effect (Δ): -0.0001 (n/a; negative = idea better)
Wins: 4 / 8 paired seeds
p-value: 0.5844 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Not confirmed ✗
Practical effect: No effect
Baseline tuning: swept over 0 configs

Limitations:

Only the official supervised dynamics benchmark was tested. PPO, online action-time shielding, closed-loop multi-step rollouts, rejection rates, safety violations over episodes, Jacobian estimation, disturbance shifts, and computational overhead were not tested. The implementation uses a differentiable reachable-angle penalty as a training proxy rather than a deployed action-time shield.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 2, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented the Adaptive Zonotope Safety Shield on the structurally matched official dynamics track using rnn_small, online residual center/radius updates, and a reachable-set training penalty. Test MSE decreased slightly from 0.0027758953 to 0.0026812993, but the paired permutation test was non-significant (p=0.58435), so the benchmark criterion for a win was not met. The mechanism signature was not confirmed because the learned radius was very large and all reachable intervals were unsafe.

Baseline: lr=0.01, 10 epochs; mean test MSE=0.0027758953, std=0.0017084856
Idea: lr=0.01, shield_lambda=0.06, 10 epochs; mean test MSE=0.0026812993, std=0.000926?; paired delta=-0.0000945959, 4/8 wins, p=0.58435
Math: 1000 affine zonotope cases; containment disagreements=0; maximum sampled support gap=0.0
Mechanism: Held-out residual coverage=1.0, learned radius q=3.66355, unsafe reachable-interval rate=1.0, confirmed=false

Benchmark result