Joint Modeling for Stochastic Interventions

Integrate this into a world model, offline-RL transition model, or causal representation module whenever actions, perturbations, or latent interventions are stochastic. Add three heads sharing an encoder of context \(c\): an intervention-density head \(q_{\theta}(x_{\mathrm{do}}\mid c)\), a mediator head \(p_{\phi}(m\mid x_{\mathrm{do}},c)\), and an outcome head \(p_{\phi}(y\mid x_{\mathrm{do}},m,c)\). For observed intervention samples, train the joint negative log-likelihood. For prediction under a desired intervention policy, draw \(S\) samples \(x_{s}\sim q_{\theta}\), then draw \(m_{s}\sim p_{\phi}(m\mid x_{s},c)\), and average outcome distributions or samples: \(\hat p(y\mid c)=S^{-1}\sum_{s}p_{\phi}(y\mid x_{s},m_{s},c)\). Pseudocode: encode \(c\); sample or observe \(x_{\mathrm{do}}\); compute mediator and outcome log likelihoods; optimize their sum jointly; at evaluation, Monte Carlo marginalize over the intervention and mediator rather than conditioning on a mediator value alone. The paper contributes the singular-versus-nonsingular distinction; estimate the intervention density empirically with a normalizing-flow or Gaussian head and test calibration using held-out intervention samples. The first cheap experiment should use a synthetic SCM with \(X_{\mathrm{do}}\sim\mathcal{N}(0,1)\), \(M=X_{\mathrm{do}}+U\), and \(Y=M+X_{\mathrm{do}}+V\), where \(U,V\) are independent Gaussian noises. Compare the proposed joint model against a baseline regressor trained on \((M,Y)\) that treats intervention as fixed or ignores it. Evaluate expected outcome under shifted intervention variance and under mediator selection \(M\approx m\). Success is lower interventional negative log likelihood, calibrated uncertainty, and reduced bias in estimated \(E[Y]\) under intervention-distribution shift.