arXiv:2607.15450 2026 Training 1 ideas extracted · analyzed Aug 30, 2026
The paper studies a practical prediction-only regime in which the original labeled data and teacher parameters are unavailable, but the teacher can be queried on fresh covariates. Its transferable mechanism is prediction mixing: train a new predictor on teacher pseudo-labels, then combine teacher and student outputs instead of trusting either predictor alone. A small labeled calibration set can estimate the mixing coefficient without retraining either model, making this a lightweight post-training adaptation method for distribution shift or model improvement. The first implementation should target regression, where the optimal scalar mixing coefficient has a closed-form least-squares solution.
Use fresh unlabeled covariates to train a frozen-teacher student against pseudo-labels, then form an affine combination of teacher and student predictions. Estimate the combination weight on a small independent labeled calibration set, requiring no access to the teacher training data and no additional teacher or student fitting.