{
 "artifacts": null,
 "category": "training",
 "description": "Use the corruption channel's information-loss rate to choose diffusion training weights rather than relying only on signal-to-noise heuristics. The conditional-score floor measures where the noisy observation still carries recoverable information about the clean data, allowing training compute to be concentrated on informative time regions.",
 "formulas_latex": [
  "$$X_\\tau=Y+\\sqrt{\\tau}\\,\\varepsilon,\\qquad \\varepsilon\\sim\\mathcal{N}(0,I)$$",
  "$$\\frac{d}{d\\tau}I(Y;X_\\tau)=-\\frac{1}{2\\tau^2}\\,\\mathbb{E}\\left[\\|Y-\\mathbb{E}[Y\\mid X_\\tau]\\|^2\\right]$$",
  "$$F(\\tau)=\\mathbb{E}\\left[\\operatorname{tr}\\operatorname{Cov}(\\mathbf{s}_{\\rm cond}(X_\\tau;Y)\\mid X_\\tau)\\right]=\\frac{1}{\\tau^2}\\,\\mathbb{E}\\left[\\|Y-\\mathbb{E}[Y\\mid X_\\tau]\\|^2\\right]=-2\\frac{d}{d\\tau}I(Y;X_\\tau)$$",
  "$$w_{\\rm info}(t)=\\frac{1/(\\widehat F_t+\\delta)}{\\mathbb{E}_{u\\sim p_0}[1/(\\widehat F_u+\\delta)]},\\qquad p_{\\rm info}(t)\\propto p_0(t)(\\widehat F_t+\\delta)$$"
 ],
 "id": 2714,
 "implementation": "(1) Modify the diffusion time sampler and scalar loss multiplier, not the network or reverse sampler, for the first experiment. Choose between sampling t from an information-flow density or retaining the baseline sampler while multiplying the DSM loss by an inverse-floor weight. (2) Construct a lookup table over K discrete time or noise bins using held-out clean samples. For each bin, generate noisy x_i and use a clean-data bank y_j. Compute pi_ij=softmax_j(-||x_i-alpha*y_j||²/(2*sigma²)), posterior mean mu_i=sum_j pi_ij*y_j, posterior MMSE m_i=sum_j pi_ij||y_j-mu_i||², and Fhat_k=alpha²/sigma⁴ times the batch mean of m_i. Smooth Fhat_k with an exponential moving average. Set delta to 0.01 times the median positive Fhat. For information-flow sampling use p_info(k) proportional to p0(k)(Fhat_k+delta), implemented with an alias table. If preserving the original objective is important, multiply each sampled loss by the importance ratio p0(k)/p_info(k), clipped to [0.1,10]. For inverse-floor weighting use w_info(k) proportional to 1/(Fhat_k+delta), normalized to mean one. (3) The I-MMSE identity and relation between F and mutual-information decay are mathematical inputs; Fhat, empirical posterior estimates, smoothing, and clipping are engineering approximations. (4) First test on a 2D mixture, then a 10M-parameter CIFAR-10 DDPM, comparing uniform/log-SNR sampling, p_info sampling, and inverse-floor weighting. Measure marginal-score error, FID, gradient variance, and loss versus FLOPs. Success is faster score-error reduction or better FID at equal compute without unstable gradient variance.",
 "math_summary": "For the additive Gaussian channel X_tau=Y+sqrt(tau) epsilon, where tau is noise variance, epsilon∼N(0,I), and I(Y;X_tau) is mutual information, the I-MMSE identity gives dI(Y;X_tau)/d_tau=-(1/(2 tau²))E||Y-E[Y|X_tau]||². The Gaussian conditional-score floor is F(tau)=tau^{-2}E tr Cov(Y|X_tau)=tau^{-2}E||Y-E[Y|X_tau]||², so F(tau)=-2 dI(Y;X_tau)/d_tau. For general X_t=alpha_tY+sigma_t epsilon, the floor is F_t=(alpha_t²/sigma_t⁴)E tr Cov(Y|X_t). Here E[Y|X_t] is the posterior mean and Cov(Y|X_t) is posterior covariance. An information-flow sampler can choose time density proportional to F_t, while an inverse-floor multiplier can normalize per-time learning difficulty.",
 "math_tags": [
  "information-theory",
  "probability",
  "statistics",
  "geometry"
 ],
 "ml_areas": [
  "diffusion",
  "curriculum",
  "loss",
  "training"
 ],
 "paper": {
  "arxiv_id": "2608.23916",
  "arxiv_url": "https://arxiv.org/abs/2608.23916",
  "summary_what_math_gives_to_ml": "The paper identifies an exact irreducible loss component in denoising score matching: the conditional-score regression loss exceeds the marginal-score loss by the conditional variance of the target. This variance is the trace of a Fisher–Rao metric and, for Gaussian corruption, can be written using the posterior covariance of the clean sample given the noisy sample. The most actionable transfer is to subtract or normalize this analytically estimated floor when comparing models, selecting noise weights, or constructing curricula, preventing schedule-dependent noise from being mistaken for model quality. A second opportunity is information-flow weighting, using the identity between the floor and mutual-information decay to allocate training effort where the corruption channel is most informative.",
  "title": "The Loss Floor of Denoising Score Matching: Fisher Geometry from Schrödinger Bridges",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 6,
  "usefulness": 6
 },
 "solves": [
  "speedup",
  "accuracy",
  "sample-efficiency"
 ],
 "title": "Information-flow DSM weighting",
 "url": "https://synthcore.org/idea/2714/information-flow-dsm-weighting",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}