Persistent Cross Entropy

What the math gives to ML

The paper introduces a principled way to compare probability measures living on different event spaces by retaining probability mass only where the second object explains the first, and assigning the remainder to an unexplained event. Its key transferable asset is a directional cross-entropy: unlike a symmetric distance, it can measure which structures present in a teacher are missing from a student. This suggests a topology-aware distillation regularizer that compares persistence diagrams of intermediate representations and explicitly penalizes teacher topological features that the student cannot explain. The construction is especially useful when teacher and student have different widths, resolutions, or numbers of detected topological events.