{
 "artifacts": [
  {
   "name": "bench_exp.py",
   "url": "https://synthcore.org/code/1046/bench_exp.py"
  },
  {
   "name": "bench_report.json",
   "url": "https://synthcore.org/code/1046/bench_report.json"
  },
  {
   "name": "fisher_floor_dsm.py",
   "url": "https://synthcore.org/code/1046/fisher_floor_dsm.py"
  },
  {
   "name": "fisher_floor_experiment.py",
   "url": "https://synthcore.org/code/1046/fisher_floor_experiment.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1046/report.md"
  },
  {
   "name": "report_bench_2026-09-01T211550.md",
   "url": "https://synthcore.org/code/1046/report_bench_2026-09-01T211550.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/1046/results.json"
  },
  {
   "name": "verify_floor_dsm.py",
   "url": "https://synthcore.org/code/1046/verify_floor_dsm.py"
  }
 ],
 "category": "training",
 "description": "Replace raw denoising score-matching loss reports and weighting decisions with a floor-corrected loss that removes the conditional-target variance intrinsic to the corruption process. This makes models trained under different noise schedules comparable and can produce a lower-variance validation signal for checkpoint selection.",
 "download_zip": "https://synthcore.org/download/1046",
 "formulas_latex": [
  "$$\\mathbf{s}_{\\rm cond}(\\mathbf{x},t;\\mathbf{y})=\\nabla_{\\mathbf{x}}\\log q(\\mathbf{x},t\\mid\\mathbf{y}),\\qquad \\mathbf{s}(\\mathbf{x},t)=\\nabla_{\\mathbf{x}}\\log P_t(\\mathbf{x})$$",
  "$$\\mathbb{E}\\left[\\|\\mathbf{s}_\\theta(\\mathbf{X}_t,t)-\\mathbf{s}_{\\rm cond}(\\mathbf{X}_t,t;\\mathbf{Y})\\|^2\\right]=\\mathbb{E}\\left[\\|\\mathbf{s}_\\theta(\\mathbf{X}_t,t)-\\mathbf{s}(\\mathbf{X}_t,t)\\|^2\\right]+\\mathbb{E}\\left[\\operatorname{tr}\\operatorname{Cov}(\\mathbf{s}_{\\rm cond}(\\mathbf{X}_t,t;\\mathbf{Y})\\mid\\mathbf{X}_t)\\right]$$",
  "$$X_t=\\alpha_tY+\\sigma_t\\varepsilon,\\quad \\mathbf{s}_{\\rm cond}(x,t;y)=\\frac{\\alpha_t y-x}{\\sigma_t^2},\\quad F_t=\\frac{\\alpha_t^2}{\\sigma_t^4}\\,\\mathbb{E}\\left[\\operatorname{tr}\\operatorname{Cov}(Y\\mid X_t)\\right]$$",
  "$$\\mathcal{L}_{\\rm corr}=\\int w(t)\\left(\\mathbb{E}\\|s_\\theta(X_t,t)-s_{\\rm cond}(X_t,t;Y)\\|^2-F_t\\right)\\,dt$$"
 ],
 "id": 2713,
 "implementation": "(1) Integrate this at diffusion-model validation and loss-weighting time, immediately after computing the per-example DSM squared error ||s_theta(x_t,t)-s_cond||². Initially leave the score network unchanged and use the same noise schedule and time weighting as the baseline. For Gaussian corruption x=alpha*y+sigma*epsilon, estimate the floor on each minibatch using an empirical clean-data prior. For each noisy query x_i and bank sample y_j, compute logits ell_ij=-||x_i-alpha*y_j||²/(2*sigma²), posterior weights pi_ij=softmax_j(ell_ij), posterior mean mu_i=sum_j pi_ij*y_j, and posterior covariance trace v_i=sum_j pi_ij||y_j-mu_i||². Set Fhat_i=(alpha²/sigma⁴)v_i and subtract the minibatch mean of Fhat from the raw loss; stop gradients through Fhat. For high-dimensional data, use a random memory bank and compute only the covariance trace, not the full matrix. Pseudocode: sample y,t,epsilon; form x=alpha(t)y+sigma(t)epsilon; target=(alpha*y-x)/sigma²; raw=||net(x,t)-target||²; compute pi over the bank; floor=alpha²/sigma⁴ times weighted variance; optimize mean(w(t)*(raw-floor)). The decomposition and Gaussian floor formula come from the mathematics; the finite-bank posterior is an approximation. First test on MNIST or CIFAR-10 with small DDPMs under cosine and linear schedules. Compare raw loss, corrected loss, and FID or held-out score error. Success means corrected loss ranks checkpoints and schedules more consistently with sample quality and removes schedule-dependent additive offsets.",
 "math_summary": "Let Y be a clean data sample, X_t a noisy sample drawn from corruption kernel q_t(x|y), s_theta(x,t) the learned score, s_cond(x,t;y)=∇_x log q_t(x|y) the conditional score, and s(x,t)=∇_x log p_t(x) the marginal score of X_t. Because E[s_cond(X_t,t;Y)|X_t=x]=s(x,t), the conditional-variance decomposition gives E||s_theta(X_t,t)-s_cond(X_t,t;Y)||² = E||s_theta(X_t,t)-s(X_t,t)||² + E tr Cov(s_cond(X_t,t;Y)|X_t). The second term F_t is the irreducible Fisher–Rao floor, where Cov denotes conditional covariance and tr its trace. For Gaussian corruption X_t=α_tY+σ_tε with ε∼N(0,I), s_cond=(α_tY-X_t)/σ_t², so F_t=(α_t²/σ_t⁴)E tr Cov(Y|X_t). The corrected objective subtracts the schedule-dependent floor from the observed DSM loss.",
 "math_tags": [
  "geometry",
  "probability",
  "statistics",
  "information-theory"
 ],
 "ml_areas": [
  "diffusion",
  "loss",
  "training"
 ],
 "paper": {
  "arxiv_id": "2608.23916",
  "arxiv_url": "https://arxiv.org/abs/2608.23916",
  "summary_what_math_gives_to_ml": "The paper identifies an exact irreducible loss component in denoising score matching: the conditional-score regression loss exceeds the marginal-score loss by the conditional variance of the target. This variance is the trace of a Fisher–Rao metric and, for Gaussian corruption, can be written using the posterior covariance of the clean sample given the noisy sample. The most actionable transfer is to subtract or normalize this analytically estimated floor when comparing models, selecting noise weights, or constructing curricula, preventing schedule-dependent noise from being mistaken for model quality. A second opportunity is information-flow weighting, using the identity between the floor and mutual-information decay to allocate training effort where the corruption channel is most informative.",
  "title": "The Loss Floor of Denoising Score Matching: Fisher Geometry from Schrödinger Bridges",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 7,
  "usefulness": 7
 },
 "solves": [
  "accuracy",
  "generalization"
 ],
 "title": "Fisher-floor-corrected DSM",
 "url": "https://synthcore.org/idea/2713/fisher-floor-corrected-dsm",
 "verification": {
  "benchmark": {
   "track": "multitoken_diffusion",
   "model": "mlp_tiny",
   "n_seeds": 8,
   "metric_direction": "lower is better",
   "comparison": {
    "baseline_mean": 22.182694673538208,
    "idea_mean": 22.182694673538208,
    "delta_mean": 0,
    "per_seed_diffs": [
     0,
     0,
     0,
     0,
     0,
     0,
     0,
     0
    ],
    "idea_wins": 0,
    "n_pairs": 8,
    "p_value": 1,
    "mde": 0,
    "mde_rel_pct": 0,
    "verdict": "no measurable effect",
    "system_worked": false
   },
   "baseline": {
    "best_cfg": {
     "epochs": 15,
     "lr": 0.003
    },
    "sweep": [
     {
      "cfg": {
       "epochs": 15,
       "lr": 0.001
      },
      "mean": 108.99711322784424
     },
     {
      "cfg": {
       "epochs": 15,
       "lr": 0.003
      },
      "mean": 22.182694673538208
     },
     {
      "cfg": {
       "epochs": 15,
       "lr": 0.006
      },
      "mean": 24.329830408096313
     }
    ],
    "full": {
     "mean": 22.182694673538208,
     "std": 4.590790464181882,
     "per_seed": [
      22.129037857055664,
      15.119169235229492,
      16.542070388793945,
      19.289772033691406,
      25.992231369018555,
      26.214162826538086,
      29.02522087097168,
      23.149892807006836
     ],
     "n": 8
    }
   },
   "idea": {
    "mean": 22.182694673538208,
    "std": 4.590790464181882,
    "per_seed": [
     22.129037857055664,
     15.119169235229492,
     16.542070388793945,
     19.289772033691406,
     25.992231369018555,
     26.214162826538086,
     29.02522087097168,
     23.149892807006836
    ],
    "n": 8
   },
   "mechanism_signature": {
    "confirmed": true,
    "predicted_vs_observed": {
     "observed_abs_delta_max": 0,
     "observed_metric_delta_mean": 0,
     "predicted_gradient_change": 0
    },
    "prediction": "detached floor subtraction leaves parameter gradients unchanged under identical weighting"
   },
   "custom_track": {
    "domain": "diffusion-sampling",
    "file": "bench_exp.py",
    "name": "multitoken_diffusion"
   }
  },
  "mechanism_confirmed": true,
  "peer_reviewed": false,
  "practical_verdict": "no_effect",
  "stage1_mechanism_check": {
   "worked": true,
   "confidence": 9,
   "verdict": "Built and ran a reproducible Fisher-floor DSM toy experiment for a 1D Gaussian-mixture prior with exact posterior floors and a finite memory-bank estimator. The decomposition residual was 0.39%, alpha scaling matched the predicted quadratic law within 4.7%, alpha=0 gave exactly zero floor, and bank error scaled with slope -0.461 versus the predicted -0.5. Pooled checkpoint rank correlation with ideal score error improved from 0.615 for raw DSM to 0.937 for corrected DSM, while the cross-schedule loss offset fell from 1.680 to -0.019; this supports schedule comparability but does not establish improved sample quality.",
   "metrics": {
    "baseline": "Raw DSM pooled checkpoint-vs-ideal rank correlation 0.615; mean cross-schedule loss offset 1.680.",
    "idea": "Corrected DSM pooled rank correlation 0.937; mean cross-schedule offset -0.019; decomposition relative error 0.0039; alpha quadratic-fit maximum relative error 0.0468; bank log-log error slope -0.461 versus -0.5 predicted; alpha=0 floor 0.0."
   },
   "how_to_run": "python3 fisher_floor_experiment.py",
   "files": [
    "fisher_floor_experiment.py",
    "results.json"
   ],
   "limitations": "Only a low-dimensional synthetic Gaussian-mixture experiment was tested. No MNIST/CIFAR diffusion network, training-speed or FLOP comparison, FID, or real checkpoint-selection experiment was performed. The finite-bank estimator has sampling noise and can produce slightly negative corrected empirical losses."
  },
  "status": "mech_ok_no_baseline",
  "status_label": "Mechanism confirmed, baseline not beaten",
  "updated_at": "2026-09-01T21:15:50",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}