Sharp Continuity of Petz and Sandwiched Rényi Conditional Entropies

arXiv:2608.04947 2026 Regularization 1 ideas extracted · analyzed Aug 31, 2026

What the math gives to ML

The paper supplies a sharp, dimension-aware continuity modulus for optimized Rényi conditional entropy under trace-distance perturbations. Its transferable asset is not the quantum formalism itself, but the explicit worst-case entropy budget \(\Gamma_{\alpha,D}(\delta)\), including saturation at \(\log D\) and an equality construction showing that the bound is tight. This can turn entropy regularization in classifiers, attention maps, or discrete latent modules into a perturbation-aware objective with a mathematically calibrated margin rather than an arbitrary robustness coefficient. The most practical initial test is to apply the bound to normalized neural probability vectors or attention distributions under augmentation, quantization, or dropout perturbations.

Ideas from this paper

Unverified 2026

Rényi entropy robustness margin

Add a certified perturbation margin to entropy-based losses so that the desired entropy remains valid after input augmentation, quantization, dropout, or attention noise. Instead of treating the entropy change caused by a perturbation as an uncontrolled empirical quantity, use the sharp modulus \(\Gamma_{\alpha,D}(\delta)\) to enforce a worst-case-safe entropy target.

Useful5/10
Difficulty3/10
Novelty6/10
Paper: Sharp Continuity of Petz and Sandwiched Rényi Conditional Entropies arXiv:2608.04947