Missingness-as-a-Label Signal

Integrate this at the classifier loss, after the model produces logits for every training example. Let logits l_j=f_theta(x_j), probabilities p_j=softmax(l_j), and m_j=1 when z_j is unavailable. Compute entropy H_j=-sum_k p_{jk} log(p_{jk}+epsilon). Introduce two trainable scalar parameters a and b, and compute q_j=sigmoid(a+b*H_j). For each labeled item, use the ordinary cross-entropy plus the missingness term: loss_j=-log p_{j,z_j}-log(1-q_j). For each unlabeled item, use loss_j=-log q_j. Add lambda*(a^2+b^2), and optionally constrain b>=0 with b=softplus(beta) if the data-generation hypothesis is that uncertainty increases missingness. Compact pseudocode is: logits=model(x); p=softmax(logits); H=-(p*log(p+eps)).sum(-1); q=sigmoid(a+b*H); loss=((1-m)*CE(logits,z) - m*log(q+eps) - (1-m)*log(1-q+eps)).mean()+lambda*(a*a+b*b). Backpropagate through both p and H; do not detach the entropy in the first experiment, because the proposed information transfer comes from gradients through the posterior. The paper's exact mathematical ingredient is the posterior-based missingness likelihood; the neural approximation is the entropy-logistic link. Estimate a and b jointly, but also run an ablation fitting q from detached entropy to distinguish classifier improvement from mere missingness calibration. Start with a two-layer MLP on MNIST or CIFAR-10 using a fixed labeled/unlabeled split generated by q*=sigmoid(a*+b*H_true), where H_true is computed from a pretrained teacher or from class difficulty, and compare ordinary supervised cross-entropy, standard entropy-minimization, and the joint objective at equal numbers of observed labels. Measure test accuracy, calibration, negative log-likelihood, and performance as the labeled fraction changes. The expected signal is improved accuracy or NLL when missingness is genuinely uncertainty-dependent, with no gain or a possible degradation under random missingness; also verify that b recovers the sign of the synthetic mechanism and that the model does not simply drive all entropies to zero.