ML: Sparsity

Shifted-Mask Defect Regularization

Represent a learned sparse attention or routing pattern as a graph and penalize its second-moment defect, which measures distance from a shifted family and therefore from nested, threshold-like neighborhoods. At inference, optionally replace the learned mask by a nearby shifted mask to obtain more structured sparse indexing and predictable routing patterns.