arXiv:2607.19004 2026 Regularization 1 ideas extracted · analyzed Aug 30, 2026
The paper gives a sharp robustness boundary for sampling from approximate scores: controlling only an L^p average score error is insufficient for universally unbiased sampling, while sufficiently strong moment-generating-function control is the relevant tractability condition. Its constructive lower-bound instance hides a large displacement between two strongly log-concave Gaussians inside a smooth score perturbation whose error is concentrated in a rare tail region. For diffusion and score-based models, this suggests replacing ordinary mean-squared score matching alone with explicit sub-Gaussian tail control and testing samplers against rare-region score failures. The most direct transfer is an MGF-based score regularizer and certification diagnostic, rather than a new sampler.
Augment diffusion score matching with a penalty on the exponential moment of the score residual, targeting the sub-Gaussian error regime identified as necessary for tractable sampling. This penalizes rare, catastrophic score errors much more strongly than an L2 loss and should improve robustness of reverse-time sampling in low-density regions.