Empirical-Likelihood Distributional Target
Implementation & benchmark of arXiv:2608.19423 — Shape-Preserving Covariate Adjustment via Empirical Likelihood in Randomized Experiment
Source paper: Shape-Preserving Covariate Adjustment via Empirical Likelihood in Randomized Experiment arXiv:2608.19423 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Construct one empirical-likelihood-weighted outcome distribution per treatment or domain group, with weights chosen to match the global mean of selected covariates exactly. Use this shared weighted empirical measure as the target for a neural CDF, survival, or quantile head rather than fitting separately adjusted targets at each threshold or quantile. The target is automatically a valid probability distribution, so its CDF is monotone and its quantiles cannot cross.
Formulas
Mathematical statement
The paper defines, for treatment arm j, probabilities p_{ij} by maximizing the empirical log likelihood subject to p_{ij} >= 0, sum_{i:A_i=j} p_{ij}=1, and sum_{i:A_i=j} p_{ij} W_i = mu_W, where A_i is the group or treatment label, Y_i is the outcome, W_i in R^d is the adjustment covariate, mu_W=E[W] is the target covariate mean, and n_j is the number of observations in arm j. The optimization is strictly concave in the probabilities over the feasible set. Its Lagrange-stationary solution can be parameterized as p_{ij}=1/[n_j(1+lambda_j^T(W_i-mu_W))], where lambda_j in R^d is chosen so that sum_{i:A_i=j} p_{ij}(W_i-mu_W)=0 and all denominators are positive. The resulting weighted empirical measure is P_hat_j=sum_{i:A_i=j}p_{ij} delta_{(Y_i,W_i)}. Its outcome CDF is F_hat_j(t)=sum_{i:A_i=j}p_{ij} 1{Y_i<=t}; because p_{ij}>=0 and sum_i p_{ij}=1, F_hat_j is a valid nondecreasing CDF with limits 0 and 1. The survival target is S_hat_j(t)=1-F_hat_j(t), and the weighted quantile is Q_hat_j(tau)=inf{x:F_hat_j(x)>=tau}. The neural adaptation trains F_theta(t|W,A=j) against F_hat_j(t) or Q_hat_j(tau), while optionally enforcing monotonicity in t through positive CDF increments.
Implementation notes
(1) Integration point: use this as an auxiliary target and weighting mechanism for a neural distributional predictor. Let the model output CDF values on a fixed grid t_1<...<t_K. Prefer a monotone parameterization: output nonnegative increments d_k=softplus(z_k), compute cumulative_sum(d), and map or normalize the cumulative values so the output is a valid CDF. Precompute one weight vector p_j for every treatment or domain group using training covariates W and target mean mu_W equal to the pooled training mean. (2) Pseudocode: compute mu_W=mean(W); for each group j initialize lambda_j=0; repeatedly calculate q_i=1+lambda_j^T(W_i-mu_W), p_i=1/(n_j q_i), and residual r=sum_i p_i(W_i-mu_W); use a damped Newton update with Jacobian H=-sum_i p_i^2(W_i-mu_W)(W_i-mu_W)^T; backtrack whenever any q_i<=epsilon; renormalize p after convergence. For each minibatch, gather F_hat_j(t_k)=sum_{i:A_i=j}p_ij 1[Y_i<=t_k] from cached group-level weighted outcome histograms, then minimize the displayed CDF loss together with the ordinary prediction loss. (3) The mathematical computation is the constrained empirical-likelihood solution and the guarantee that the target CDF is monotone; estimate mu_W from pooled data, and choose covariates and grid points empirically. Monitor the maximum covariate imbalance after weighting and clip or regularize lambda only as a numerical safeguard. (4) First experiment: train a small MLP or Transformer distributional head on IHDP, ACIC, or a synthetic randomized-treatment dataset with deliberately different covariate distributions across arms. Compare ordinary unweighted CDF or Beta-NLL training, inverse-frequency weighting, and the empirical-likelihood target. Measure CDF crossing or monotonicity violations, integrated Brier or CDF error, weighted quantile pinball loss, calibration, and treatment-arm imbalance of residuals. Success is fewer crossing violations and lower integrated distributional error at equal parameter count, especially under covariate imbalance; also verify that cached weights do not create unstable gradients or excessive effective-sample-size collapse.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.