Policy-Coupled Robust Action Selection

Integrate this as a decision head on top of a contextual bandit, offline-RL, or treatment-effect model. First train one outcome model $\hat m_a(x)$ per action, either as separate heads sharing a neural encoder or as a world model predicting $(x,a)\mapsto Y(a)$. Reserve a calibration split. For every calibration context, compute all action predictions and define a candidate interval $C_q(x,a)=[\hat m_a(x)-q,\hat m_a(x)+q]$ for each radius $q$ in a sorted grid of residual values. Compute the lower endpoint of utility over the interval; for monotone decreasing utility this is often $u(a,\hat m_a(x)-q)$, but implement the infimum by evaluating both endpoints or a small outcome grid. Select $\pi_q(x)=\arg\max_a\inf_y u(a,y)$, with deterministic tie-breaking. On calibration records, evaluate whether the observed outcome is inside the set for the selected action. If the data are randomized with known propensity $p_i=\Pr(A_i=a_i\mid X_i)$, use the inverse-propensity estimate $w_i=\mathbf 1\{a_i=\pi_q(x_i)\}/p_i$ in the displayed policy-coupled error; clip propensities to avoid exploding variance. Choose the smallest $q$ whose error estimate plus an empirical-Bernstein confidence radius is at most $\alpha$; use a Bonferroni correction over the radius grid. At inference, return the robust action and optionally its worst-case utility margin. The max-min rule and policy-coupled event come directly from the paper; residual intervals, propensity weighting, confidence correction, and grid search are practical approximations. The first experiment should use a synthetic contextual bandit with two actions and known heterogeneous potential outcomes, then a small MLP compared with greedy prediction, independent per-action conformal intervals, and pessimistic lower-confidence-bound selection. Measure realized utility, policy-coupled coverage, and regret at equal calibration size. Success means coverage remains at or above $1-\alpha$ while utility exceeds independent conformal selection, especially when actions have different outcome variances or treatment effects.