A Complete Characterization of Tensorizable $f$-divergences
arXiv:2608.28556
2026
Training
1 ideas extracted · analyzed Sep 2, 2026
What the math gives to ML
The paper identifies tensorization as a strong compositional constraint on distribution discrepancies: for product measures, the global divergence is determined solely by marginal divergences through a continuous multi-affine map. This provides a principled alternative to estimating a divergence on the full product space, especially for models with factorized, autoregressive, or independently parameterized outputs. The most actionable transfer is to replace noisy joint-distribution divergence estimates with exact marginal aggregation for KL-, chi-squared-, or Hellinger-type objectives, while explicitly preserving the correct product-measure semantics.
Ideas from this paper
Unverified
2026
When a neural model defines a product distribution over coordinates, compute the discrepancy to a target product distribution from per-coordinate divergences using the exact tensorization rule instead of sampling full vectors and estimating a joint divergence. Implement KL, chi-squared, and squared-Hellinger variants as drop-in losses, with an optional learned choice among these mathematically tensorizable families.
Useful6/10
Difficulty3/10
Novelty6/10