Diffusion models recover accurate mixture weights despite score function insensitivity

What the math gives to ML

The paper identifies a concrete failure mode in diffusion training: the clean-data score can be nearly insensitive to mixture weights even when intermediate noisy scores contain strong information about them. Its diffusion score sensitivity index (DSSI) measures a lower bound on how much the DSM objective changes under perturbations of a target distribution parameter, and this quantity depends on the noise schedule. A transferable method is therefore to select or regularize the noise schedule using an empirical minimum DSSI, favoring schedules that make important parameters identifiable at intermediate times rather than merely optimizing generic denoising quality.