✗ Failed on benchmark 2026

Composed Hamilton-Jacobi Reachability Critics

Implementation & benchmark of arXiv:2608.18060 — Extending and Unifying the Fundamental Tasks of Hamilton-Jacobi Reachability Analysis

Usefulness8/10

Difficulty6/10

Novelty7/10

Dynamics Stability Accuracy Sample efficiency Rl World model Training dynamics Loss Control theory Pde Dynamical systems Optimization

Source paper: Extending and Unifying the Fundamental Tasks of Hamilton-Jacobi Reachability Analysis arXiv:2608.18060 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Attach one neural value head to each generalized reach-avoid subtask and compose these heads into a critic for sequential or timed temporal-logic goals. The policy is trained to increase the composed value while an auxiliary Hamilton-Jacobi residual trains each local head against the learned or known dynamics. This replaces a single poorly conditioned long-horizon objective with short-horizon certificates whose composition has an explicit logical meaning.

Formulas

$$\dot{x}(t)=f(x(t),u(t),d(t),t),\qquad \partial_tV_i(x,t)+\min_{u\in\mathcal U}\max_{d\in\mathcal D}\nabla_xV_i(x,t)^\top f(x,u,d,t)=0.$$

$$\mathcal{R}_i(x,t)=\partial_tV_i(x,t)+\min_{u\in\mathcal U}\max_{d\in\mathcal D}\nabla_xV_i(x,t)^\top f_\theta(x,u,d,t),\qquad \mathcal L_{\mathrm{HJ}}=\sum_i\mathbb E[|\mathcal R_i|^2].$$

$$V_{1\rightarrow2}(x,t)=\sup_{\tau\in[t,T]}\min\{V_1(x,t;\tau),V_2(x,\tau)\},\qquad V_{\wedge}(x,t)=\min_iV_i(x,t),\quad V_{\vee}(x,t)=\max_iV_i(x,t).$$

$$u^*(x,t)=\arg\min_{u\in\mathcal U}\max_{d\in\mathcal D}\nabla_xV_{\mathrm{comp}}(x,t)^\top f_\theta(x,u,d,t).$$

Mathematical statement

Let $x\in\mathbb{R}^{n}$ be the state, $u\in\mathcal{U}$ the policy control, $d\in\mathcal{D}$ an adversarial disturbance, and $\dot{x}=f(x,u,d,t)$ the dynamics. For subtask $i$, let $G_i(x,t)$ be a signed target function, positive inside the target, and $A_i(x,t)$ a signed avoid function, positive inside the forbidden set. A differentiable GRA value $V_i(x,t)$ is trained to satisfy the robust Hamilton-Jacobi equation $\partial_tV_i+\min_{u\in\mathcal U}\max_{d\in\mathcal D}\nabla_xV_i^\top f(x,u,d,t)=0$ away from target and avoid boundaries, with obstacle boundary conditions encoding target attainment and avoidance. For a sequential specification requiring subtask 1 then subtask 2, use the value composition $V_{1\rightarrow2}(x,t)=\sup_{\tau\in[t,T]}\min\{V_1(x,t;\tau),V_2(x,\tau)\}$, where $\tau$ is the switching time; for conjunction use $\min_iV_i$, and for disjunction use $\max_iV_i$. The robust policy action is selected by minimizing the local Hamiltonian, $u^*(x,t)=\arg\min_u\max_d\nabla_xV_{\mathrm{comp}}^\top f(x,u,d,t)$. The useful property is that local PDE residuals and explicit max/min compositions expose which subtask causes failure instead of hiding it in one scalar return.

Implementation notes

1. Exact integration point: use a model-based RL or neural controlled-dynamics setup. Let $f_\theta$ be either a known simulator, a learned differentiable dynamics model, or a neural ODE. Add $K$ MLP heads $V_{\phi_i}(x,t)$, one for each target/avoid primitive. Keep the policy $\pi_\psi(u|x,t)$ separate; during actor updates, maximize the composed value at the initial state. 2. Pseudocode: sample states $x,t$, controls $u$, and disturbances $d$; evaluate $f_\theta$ and automatic derivatives of every $V_{\phi_i}$; approximate the min/max Hamiltonian with softmin/softmax temperature $\beta$; compute $\mathcal R_i=\partial_tV_i+\operatorname{softmin}_u\operatorname{softmax}_d(\nabla_xV_i^\top f_\theta)$; apply target and avoid obstacle losses; form $V_{\mathrm{comp}}$ with soft versions of the stated max/min and a switching-time grid; update critic parameters using $\mathcal L=\mathcal L_{\mathrm{HJ}}+\lambda_b\mathcal L_{\mathrm{boundary}}+\lambda_c\mathcal L_{\mathrm{composition}}$, then update the actor toward $u^*$. 3. Computed from the paper: the GRA decomposition and max/min temporal composition. Estimated empirically: $f_\theta$, PDE derivatives, disturbance maximization, and boundary signs. 4. First cheap experiment: use a 2-D Dubins or point-mass system with bounded control and disturbance, and a timed goal sequence such as reach region A by time 1, then region B by time 2 while avoiding a disk. Compare SAC or model-predictive control against one monolithic value head and the composed critic. Measure success, minimum obstacle margin, HJ residual, and calibration of the sign of $V_{\mathrm{comp}}$. The quantitative prediction is that the composed critic's success boundary in initial-state space should agree with a gridded dynamic-programming reachability solution within 10% Hausdorff distance, while halving the temporal subtask horizon should reduce one-step composition error approximately linearly with the integration step. A second signature is a sharp empirical transition: initial states with $V_{\mathrm{comp}}>0$ should have substantially higher success probability than states with $V_{\mathrm{comp}}<0$; tune the threshold on a validation grid and report the calibration curve rather than only average return.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a self-contained composed reachability critic toy for bounded 1D dynamics, including local signed value heads, sequential max-min composition, logical min/max composition, HJ residual checking, switch-time discretization, soft composition, and a monolithic neural comparison. The mechanism manifested: the affine HJ residual was 2.7e-11, logical sign composition had zero violations, switch-grid error scaled linearly as 0.48 times the time step, and softmin error followed the predicted ln(2)/beta bound. The composed analytic critic achieved 1.000 test sign accuracy versus 0.981 for the small monolithic neural head, although this is not a full RL study.

Agent confidence: 8/10

Baseline: Monolithic MLP: 0.981 test accuracy on sequential reachability sign labels.
Idea: Analytic composed critic: 1.000 test accuracy; HJ maximum absolute residual 2.66e-11; switch-grid maximum errors 0.24, 0.12, 0.06, 0.03, 0.015, 0.0075, and 0.00375 for time steps 0.5 through 0.0078125; conjunction/disjunction sign violations 0/0; softmin errors approached ln(2)/beta.

Limitations: This is a low-dimensional analytic toy, not a learned multi-head critic trained with HJ losses, actor updates, or adversarial disturbances in an RL loop. It does not test Dubins or point-mass dynamics, obstacle-boundary training, calibration curves, Hausdorff reachability error, sample efficiency, FLOPs, or long-horizon policy success.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.18060) Sep 1, 2026

Significantly WORSE than the tuned baseline (+645.9%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0008
Idea mean: 0.0057
Effect (Δ): 0.0049 (+645.9%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±126.9%
Mechanism: Not confirmed ✗
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

The built-in dynamics track is rollout regression rather than a full reach-avoid RL environment. The MVP uses a supervised composed-value loss and consistency residual; it does not implement actor updates, adversarial disturbance optimization, explicit PDE time derivatives, obstacle boundary conditions, or policy success/Hausdorff calibration. CUDA initially failed with a cuDNN host-allocation error and the implementation fell back to CPU.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 10/10

Built a dynamics-track implementation with a two-head composed recurrent critic and ran the required 8 paired seeds with a shared baseline/idea learning-rate grid. The baseline achieved mean test MSE 0.000763, while the composed idea achieved 0.005691; paired delta was +0.004928 with permutation p=0.0081, so the idea was significantly worse. The trained models did exhibit the expected soft-min ordering signature, but that mechanism signal did not translate into better task performance.

Baseline: rnn_small standard MSE, best lr=0.006, 8-seed mean test MSE 0.0007629804 (std 0.0001532)
Idea: two local recurrent value heads with soft-min composition and consistency loss, best lr=0.006, 8-seed mean test MSE 0.005690784 (std 0.0010611)
Idea wins: 0
Mechanism signature: map[composition_leq_hard_min_fraction_mean:1 confirmed:true mean_head_gap:0.0079404 softmin_hardmin_abs_gap_mean:0.1693846]
Paired delta: idea - baseline = +0.004927803 (higher is worse)
Permutation p value: 0.0081

Benchmark result