Trimmed One-Sided Observable Domination

Integrate this as an auxiliary loss during teacher-student distillation or cross-domain fine-tuning. Let h_theta(x_i) be the student embedding for source minibatch examples x_i, and let y_j be teacher embeddings or target-domain examples. Construct a minibatch cost C_ij = ||W h_theta(x_i) - y_j|| squared divided by tau, then compute an entropic Sinkhorn coupling pi_ij with uniform marginals. Retain a mask S containing the lowest-cost pairs whose total coupling mass is at least 1-epsilon. Use K frozen teacher scalar probes g_k(y) and M trainable student probes f_m(h_theta(x)), such as linear probes or small two-layer MLPs. For every teacher probe k, compute the maximum absolute error over retained pairs for each student probe m, select the minimum over m, and then take the maximum over k. This implements the finite version of the paper's supremum-infimum-supremum expression. Use log-sum-exp for the outer maximum and softmin for the inner minimum during early training, then optionally switch to hard selections. Recompute the coupling and mask each minibatch. Stop gradients through hard top-mass selection, or use a differentiable mask q_ij = sigmoid((r-C_ij)/eta), with threshold r chosen so the weighted mass is 1-epsilon. The paper supplies the one-sided direction, retained mass, and nested max-min error; Sinkhorn temperature, probe architecture, and soft masking are engineering approximations. First test on CIFAR-10 or an Office-31 domain-adaptation split using a ResNet-18 student and a larger frozen teacher. Compare standard MSE distillation, symmetric CORAL or MMD alignment, and this loss with epsilon in {0, 0.05, 0.2}. Measure target accuracy, worst-group accuracy, sensitivity to injected outlier pairs, and gradient variance. Success means higher target accuracy and less degradation under 10 to 20 percent mismatched or corrupted pairs at comparable FLOPs.