On efficiency gains via augmenting a tiny sample with a massive auxiliary sample

What the math gives to ML

The paper identifies a concrete mechanism by which a massive auxiliary sample can improve estimation for a tiny target sample: parameterize the target distribution together with the target/auxiliary odds and fit both by full likelihood rather than relying only on inverse-probability weighting. The transferable asset is Tukey's factorization, which turns a density-ratio problem into a binary domain-classification likelihood, plus the Fisher-information decomposition showing that between-population variation contributes information about shared parameters. In neural networks this suggests training a shared representation with a small target objective and a massive auxiliary domain-likelihood objective, while preserving a target-specific head. The main experiment should test whether shared parameters and representations improve at the auxiliary-sample rate without allowing auxiliary data to overwrite target-specific behavior.