High-Dimensional Spectral Limits for Gaussian KL-Unbalanced Optimal Transport
arXiv:2608.26693
2026
Regularization
1 ideas extracted · analyzed Aug 29, 2026
What the math gives to ML
The paper develops a nonstandard Gaussian KL-unbalanced optimal-transport covariance functional whose equal-penalty form is expressed through nonlinear ridge products and log-determinants, with a positive-semidefinite extension that remains finite for singular covariances. Its most transferable asset is dimension-aware spectral regularization: in high-dimensional sample-covariance regimes, the KL penalty must scale critically as \(\tau_p\asymp p\) to control covariance noise. This suggests replacing unstable minibatch covariance matching or Bures-style feature losses with a ridge-logdet discrepancy calibrated to feature dimension. The first test should measure whether this improves representation stability and downstream accuracy when feature dimension is comparable to batch size.
Ideas from this paper
✗ Mechanism failed
2026
Add a Gaussian KL-UOT-inspired covariance discrepancy to a neural representation loss, using ridge-logdet terms that remain finite when minibatch covariance matrices are rank deficient. Set the unbalanced penalty to \(\tau=\kappa p\), where \(p\) is the feature dimension and \(\kappa\) is tuned over a small logarithmic grid, rather than using a dimension-independent covariance penalty. This directly tests the paper's claim that high-dimensional sample-covariance noise has a critical penalty…
Useful6/10
Difficulty5/10
Novelty6/10