△ Mechanism confirmed, baseline not beaten 2026

Monotone Compositional Reachability Critic

Implementation & benchmark of arXiv:2608.27654 — Exact Decomposition of Value Functions for Two-Player Games in Hamilton-Jacobi Reachability

Usefulness7/10

Difficulty5/10

Novelty7/10

Architecture Stability Accuracy Scalability Rl World model Loss Regularization Control theory Dynamical systems Pde Optimization

Source paper: Exact Decomposition of Value Functions for Two-Player Games in Hamilton-Jacobi Reachability arXiv:2608.27654 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Train separate neural value functions for primitive reachability, avoidance, or target-reaching tasks, then combine them with a coordinatewise monotone aggregator whose derivatives with respect to all primitive values are nonnegative. This transfers the paper's exact two-player decomposition condition into a modular critic architecture: adding a new target changes only one primitive critic and the aggregator, rather than requiring a new high-dimensional value function.

Formulas

$$\dot{x}(t)=f(x(t),u(t),d(t),t),\qquad x\in\mathbb R^n,\ u\in\mathcal U,\ d\in\mathcal D.$$

$$\partial_t V(x,t)+\min_{u\in\mathcal U}\max_{d\in\mathcal D}\nabla_xV(x,t)^\top f(x,u,d,t)=0.$$

$$\widehat V(x,t)=\Phi_\theta(\widehat V_1(x,t),\ldots,\widehat V_m(x,t)),\qquad \frac{\partial\Phi_\theta}{\partial z_i}(z)\ge0\quad\forall i,z.$$

$$\mathcal R_t=\{x:V(x,t)\le0\},\qquad \widehat{\mathcal R}_t=\{x:\widehat V(x,t)\le0\};\quad \text{monotone decomposition predicts }\widehat{\mathcal R}_t\approx\mathcal R_t.$$

Mathematical statement

Consider continuous-time dynamics $\dot{x}=f(x,u,d,t)$ with state $x\in\mathbb{R}^n$, control $u\in\mathcal U$, disturbance $d\in\mathcal D$, and finite horizon $t\in[0,T]$. For a value function $V(x,t)$, the two-player Hamilton-Jacobi equation is $\partial_tV+H(x,\nabla_xV,t)=0$, with Hamiltonian $H(x,p,t)=\min_{u\in\mathcal U}\max_{d\in\mathcal D}p^\top f(x,u,d,t)$. Let $V_i(x,t)$ denote primitive value functions and let the composite task value be $V(x,T)=\Phi(V_1(x,T),\ldots,V_m(x,T))$. The transferable condition is coordinatewise monotonicity: $\partial\Phi/\partial z_i\ge0$ for every primitive coordinate $z_i$. Implement $\widehat V(x,t)=\Phi_\theta(\widehat V_1(x,t),\ldots,\widehat V_m(x,t))$ with nonnegative aggregator derivatives. The property to test is preservation of the composite value and its zero-sublevel reachable set; if monotonicity is violated, the ordering of primitive task values can be reversed and the decomposition can fail.

Implementation notes

1. Exact integration point: use this as a critic for continuous-control RL, model-predictive safety filtering, or neural HJ reachability. Train one network $V_i(x,t)$ per primitive target or constraint and a small aggregator network $\Phi_\theta$ on their scalar outputs. Use a monotone lattice or positive-weight network, for example $\Phi_\theta(z)=b+\sum_k a_k\,\mathrm{softplus}(w_k^\top z+c_k)$, with $a_k\ge0$ and $w_{ki}\ge0$ enforced by softplus parameterizations. This guarantees nonnegative partial derivatives. 2. Pseudocode: sample $(x,t)$, controls, and disturbances; compute $z_i=V_i(x,t)$; set $V=\Phi_\theta(z)$; obtain $\nabla_xV$ by automatic differentiation; approximate the Hamiltonian using sampled controls and disturbances or differentiable soft min/max; minimize $|\partial_tV+H(x,\nabla_xV,t)|^2$ plus terminal error $|V(x,T)-\Phi_\theta(V_i(x,T))|^2$. Pretrain each primitive critic using a low-dimensional HJ solver or its own PDE residual. 3. Quantities computed directly are network gradients, PDE residuals, aggregator derivatives, and zero-sublevel masks. The theorem supplies the monotonicity condition; decomposition error, Hamiltonian error, and set mismatch must be estimated empirically. Use adversarial rollouts with approximate maximizing disturbances to test the game value. 4. First cheap experiment: use a two-dimensional Dubins or planar single-integrator system with bounded control and disturbance, two target disks, and one obstacle. Compare a monolithic value network, an unconstrained MLP aggregator, and the positive-weight monotone aggregator against a grid-based finite-horizon HJ solution. Measure value RMSE, PDE residual, reachable-set symmetric difference, and adversarial rollout violations. The quantitative prediction is that the monotone model's set error remains near the propagated primitive errors, while the unconstrained model fails when $\partial\Phi/\partial z_i<0$. Sweep one aggregator coefficient through zero; reachable-set mismatch and violation rate should increase sharply after a negative-derivative region appears. Target a monotone-model set IoU above 0.9 and an unconstrained residual at least 2 times larger after the sign violation.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a monotone positive-weight compositional critic and numerically tested its core claims. The aggregator preserved coordinatewise ordering in 100% of trials, primitive-value error propagated linearly with log-log exponent 0.9999, and the direct order-reversal transition occurred exactly when the swept coefficient became negative (0% reversal for c2>=0, 100% for c2<0). In the matched synthetic critic task, monotone and unconstrained aggregators performed essentially identically, so this verifies the mechanism but does not demonstrate a practical accuracy win.

Agent confidence: 8/10

Baseline: Unconstrained linear aggregator: held-out RMSE 0.03186373 and set IoU 0.92797786; learned weights [0.998989, 0.795554].
Idea: Positive-weight monotone aggregator: held-out RMSE 0.03186373 and set IoU 0.92797786; learned weights [0.998989, 0.795554]. Core checks: ordering preservation 1.0, error-bound maximum ratio 1.00073, error-scaling exponent 0.999923, and negative-coefficient order reversal 0% for c2>=0 versus 100% for c2<0.

Limitations: This is a small synthetic static value-function experiment, not a continuous-time HJ solver, adversarial rollout, RL critic, or learned nonlinear aggregator. The unconstrained model was trained on data generated by a monotone target, so it learned positive weights and had no opportunity to expose a practical generalization advantage. Runtime, scalability, and high-dimensional reachability were not tested.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.27654) Sep 2, 2026

Ahead of the baseline (−10.3%) but not statistically significant (p=0.5667, wins 4 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0055
Idea mean: 0.0049
Effect (Δ): -0.0006 (−10.3%; negative = idea better)
Wins: 4 / 8 paired seeds
p-value: 0.5667 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±41.6%
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

Only the built-in actuated-pendulum rollout regression track was tested; no PDE/HJ grid solver, adversarial disturbance rollout, target-set IoU, nonlinear monotone lattice, or larger-scale/high-dimensional reachability experiment was run. The benchmark used 400 training examples, 200 test examples, 10 epochs, and a small GRU.

How to run: python3 bench_experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 2, 2026 latest Failed ✗

Agent confidence: 9/10

The registered built-in dynamics track was run with 8 paired seeds, a tuned baseline sweep, and a parity-matched idea sweep. The monotone system had lower mean MSE, but the paired permutation test was not significant (p=0.56675; 4/8 wins), so this is not a benchmark win. The trained-model mechanism signature was confirmed: the idea had 100% nonnegative observed aggregator derivatives, while the unconstrained baseline had 0%.

Baseline: dynamics/rnn_small unconstrained aggregator; best lr=0.006, epochs=10; full mean test MSE=0.005481509841047227.
Idea: dynamics/rnn_small positive-weight monotone aggregator; best lr=0.003, epochs=10; full mean test MSE=0.0049165511154569685; paired delta=-0.0005649587255902588; p=0.56675; 4/8 wins.

Benchmark result