What Does a Discrete Diffusion Model Learn?

What the math gives to ML

The paper gives an exact CTMC interpretation of discrete-diffusion training: the negative ELBO is data entropy plus a path KL between the oracle and learned reverse processes, so the optimum is a conditional reverse jump-rate predictor rather than an ambiguously named denoiser or score. Its most transferable result is the exact change of coordinates between clean-token posteriors, bridge/cavity predictors, and reverse jump rates. For uniform token corruption, converting a neural denoiser into reverse rates through the Bayes bridge ratio should avoid the initialization divergence that occurs when denoiser probabilities are inserted directly into the uniform ELBO. The most direct experiment is to keep a standard categorical denoiser backbone but replace its sampler and ELBO-rate head with the analytically converted bridge parameterization.