A Composite Divergence Approach to Robust Multivariate Estimation under Cellwise and Casewise Contamination
arXiv:2608.18914
2026
Regularization
1 ideas extracted · analyzed Sep 1, 2026
What the math gives to ML
The paper introduces a composite density-power divergence that robustifies low-dimensional likelihood components without requiring a tractable full joint density. Its transferable asset is an explicit bounded-influence weighting mechanism: components assigned low predictive probability contribute less to the gradient, protecting training from whole-example and individual-feature corruption. A direct neural-network adaptation is to replace the usual negative log-likelihood with a weighted composite DPD loss over output heads, tokens, labels, nodes, or edges. The normalization term involving the model density raised to power 1+alpha prevents the method from degenerating into arbitrary loss clipping.
Ideas from this paper
△ Mechanism confirmed, baseline not beaten
2026
Replace a neural network's full-example negative log-likelihood by a weighted sum of density-power-divergence losses over low-dimensional predictive components. For positive tuning parameter alpha, components assigned low probability receive gradient weight proportional to the predicted probability raised to alpha, so isolated corrupted labels or feature cells cannot dominate training. The normalizing integral term preserves a proper divergence objective rather than applying uncalibrated…
Useful7/10
Difficulty4/10
Novelty6/10