{
 "artifacts": [
  {
   "name": "bench_report.json",
   "url": "https://synthcore.org/code/1141/bench_report.json"
  },
  {
   "name": "math_check.py",
   "url": "https://synthcore.org/code/1141/math_check.py"
  },
  {
   "name": "poisson_field_track.py",
   "url": "https://synthcore.org/code/1141/poisson_field_track.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1141/report.md"
  },
  {
   "name": "report_bench_2026-09-02T121801.md",
   "url": "https://synthcore.org/code/1141/report_bench_2026-09-02T121801.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/1141/results.json"
  },
  {
   "name": "run_bench.py",
   "url": "https://synthcore.org/code/1141/run_bench.py"
  },
  {
   "name": "spatial_quantile_conformal.py",
   "url": "https://synthcore.org/code/1141/spatial_quantile_conformal.py"
  }
 ],
 "category": "regularization",
 "description": "Replace a worst-case spatial residual score with the (1-gamma)-quantile of the normalized residual field, then calibrate this scalar score on held-out operator examples. At test time, inflate the predicted uncertainty field by the conformal order statistic; the guarantee targets the fraction of spatial domain covered, producing tighter bands than max-error or Bonferroni corrections.",
 "download_zip": "https://synthcore.org/download/1141",
 "formulas_latex": [
  "$$r_a(x)=\\frac{\\|\\mathcal G^\\dagger(a)(x)-\\hat{\\mathcal G}(a)(x)\\|_2}{\\hat{\\mathcal E}(a)(x)},\\qquad S(a)=\\inf\\left\\{s\\geq0:\\nu\\left(\\left\\{x\\in D:r_a(x)\\leq s\\right\\}\\right)\\geq1-\\gamma\\right\\}.$$",
  "$$k=\\min\\left\\{n,\\left\\lceil(n+1)(1-\\alpha)\\right\\rceil\\right\\},\\qquad q=S_{(k)},\\quad S_{(1)}\\leq\\cdots\\leq S_{(n)}.$$",
  "$$C_q(a)(x)=\\left\\{v\\in\\mathbb R^{d_u}:\\|v-\\hat{\\mathcal G}(a)(x)\\|_2\\leq q\\,\\hat{\\mathcal E}(a)(x)\\right\\}.$$",
  "$$\\Pr\\left[\\nu\\left(\\left\\{x\\in D:\\mathcal G^\\dagger(a)(x)\\in C_q(a)(x)\\right\\}\\right)\\geq1-\\gamma\\right]\\geq1-\\alpha.$$"
 ],
 "id": 2943,
 "implementation": "1. Integrate the method after a trained neural operator, such as an FNO or DeepONet, at inference time. The model must output a field hat u on the target grid. Add a positive uncertainty head hat e=softplus(h)+epsilon, or use a validation-derived residual scale map; conformal calibration itself is not backpropagated. 2. Pseudocode: for every held-out calibration pair (a_i,u_i), compute predictions hat u_i and scales hat e_i; calculate r_ij=||u_ij-hat u_ij||_2/(hat e_ij+epsilon); sort r_ij over spatial cells and find the weighted empirical quantile containing mass 1-gamma, producing S_i. Sort the S_i values, set k=min(n,ceil((n+1)(1-alpha))), and set q=S_(k). For a test input, return center hat u_j and radius q*hat e_j at each cell. 3. Compute spatial weights from uniform cells or quadrature weights normalized to sum one. Estimate the scale field from the uncertainty head or training-only residual statistics, and apply identical positivity, clipping, and interpolation rules during calibration and testing. 4. First experiment: train a small FNO on 2-D Darcy flow with disjoint training, calibration, and test sets. Compare against max-residual conformal calibration, pointwise split conformal, and Bonferroni intervals using identical model weights, calibration size, grid, and inference cost; evaluate alpha=0.1 and gamma in {0.01,0.05,0.10}. 5. Pre-register tests: repeated test batches must achieve domain coverage at least 1-gamma in at least 1-alpha of batches, allowing tolerance 0.02; at matched coverage, average band width should be at least 20% smaller than max-residual calibration; increasing gamma must monotonically decrease mean radius; and increasing calibration size must reduce coverage shortfall below 0.02. Measure weighted spatial hit fractions, not pointwise hit rates. The decisive ablation replaces the spatial-quantile score with max_j r_ij. Falsify the transfer if coverage is systematically below target, spatial quantiles do not tighten bands at matched coverage, or nonuniform quadrature weights break calibration.",
 "math_summary": "For an input function a in A, let u^dagger(a)(x)=G^dagger(a)(x) be the true field, hat u(a)(x)=hat G(a)(x) the neural-operator prediction, and hat e(a)(x)=hat E(a)(x)\u003e0 a scale field. With domain probability measure nu satisfying nu(D)=1, define the normalized residual r_a(x)=||u^dagger(a)(x)-hat u(a)(x)||_2/hat e(a)(x). The spatial score is S(a)=inf{s\u003e=0: nu({x in D:r_a(x)\u003c=s})\u003e=1-gamma}. Given n exchangeable calibration pairs, sort S_i=S(a_i) as S_(1)\u003c=...\u003c=S_(n), and choose q=S_(k), where k=min{n,ceil((n+1)(1-alpha))}. The band is C_q(a)(x)={v in R^{d_u}:||v-hat u(a)(x)||_2\u003c=q hat e(a)(x)}. The paper's measurable split-conformal result gives P[nu({x:G^dagger(a)(x) in C_q(a)(x)})\u003e=1-gamma] \u003e= 1-alpha, with probability over a and calibration data, under exchangeability and measurable field representatives. On a grid, nu is replaced by normalized cell or quadrature weights.",
 "math_tags": [
  "probability",
  "statistics",
  "measure-theory",
  "approximation-theory"
 ],
 "ml_areas": [
  "loss",
  "regularization",
  "world-model"
 ],
 "paper": {
  "arxiv_id": "2608.28515",
  "arxiv_url": "https://arxiv.org/abs/2608.28515",
  "summary_what_math_gives_to_ml": "The paper turns a functional prediction problem into a scalar split-conformal problem by scoring each residual field through its spatial quantile, rather than requiring simultaneous pointwise coverage everywhere. This is valuable for neural operators because it gives a calibrated statement about the fraction of a continuum or grid on which an entire predicted solution is accurate, while allowing heterogeneous spatial uncertainty through a scale field. The construction can be attached to FNO, DeepONet, or other operator surrogates without changing their training objective. The key experiment should test domain-fraction coverage and band tightness against max-residual and pointwise conformal baselines.",
  "title": "Conformal Uncertainty Quantification Guarantees for Neural Operators",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 3,
  "novelty": 5,
  "usefulness": 7
 },
 "solves": [
  "accuracy",
  "generalization",
  "stability"
 ],
 "title": "Spatial-Quantile Conformal Bands for Neural Operators",
 "url": "https://synthcore.org/idea/2943/spatial-quantile-conformal-bands-for-neural-operators",
 "verification": {
  "benchmark": {
   "track": "poisson_field_operator",
   "model": "mlp_tiny",
   "n_seeds": 8,
   "metric_direction": "lower is better",
   "comparison": {
    "baseline_mean": 0.0014413107419386506,
    "idea_mean": 0.0014413107419386506,
    "delta_mean": 0,
    "per_seed_diffs": [
     0,
     0,
     0,
     0,
     0,
     0,
     0,
     0
    ],
    "idea_wins": 0,
    "n_pairs": 8,
    "p_value": 1,
    "mde": 0,
    "mde_rel_pct": null,
    "verdict": "no measurable effect",
    "system_worked": false
   },
   "baseline": {
    "best_cfg": {
     "lr": 0.006
    },
    "sweep": [
     {
      "cfg": {
       "lr": 0.0015
      },
      "mean": 0.004578319028951228
     },
     {
      "cfg": {
       "lr": 0.003
      },
      "mean": 0.0021311483287718147
     },
     {
      "cfg": {
       "lr": 0.006
      },
      "mean": 0.0014657231222372502
     }
    ],
    "full": {
     "mean": 0.0014413107419386506,
     "std": 0.00008870037700098626,
     "per_seed": [
      0.0013521382352337241,
      0.0014695348218083382,
      0.001459407852962613,
      0.0015818115789443254,
      0.0015570472460240126,
      0.0013796440325677395,
      0.0013133999891579151,
      0.0014175021788105369
     ],
     "n": 8
    }
   },
   "idea": {
    "mean": 0.0014413107419386506,
    "std": 0.00008870037700098626,
    "per_seed": [
     0.0013521382352337241,
     0.0014695348218083382,
     0.001459407852962613,
     0.0015818115789443254,
     0.0015570472460240126,
     0.0013796440325677395,
     0.0013133999891579151,
     0.0014175021788105369
    ],
    "n": 8
   },
   "mechanism_signature": {
    "confirmed": true,
    "mean_max_event_rate": 0.99625,
    "mean_max_width": 0.24931471447160236,
    "mean_quantile_event_rate": 0.898125,
    "mean_quantile_width": 0.15621099945286587
   },
   "custom_track": {
    "domain": "pde",
    "file": "poisson_field_track.py",
    "name": "poisson_field_operator"
   }
  },
  "mechanism_confirmed": true,
  "peer_reviewed": false,
  "practical_verdict": "no_effect",
  "stage1_mechanism_check": {
   "worked": true,
   "confidence": 8,
   "verdict": "Built spatial-quantile conformal calibration with weighted spatial order statistics, finite-sample calibration indexing, positive frozen scale fields, and a reproducible synthetic operator experiment. The math sanity checks passed, including weighted quantile behavior and monotonic tightening as gamma increases. On the mini-experiment, spatial-quantile bands were 5.3% narrower at gamma=0.05 and 11.8% narrower at gamma=0.10 than max-residual bands, while the \u003e=(1-alpha) domain-coverage event rates were 0.903 and 0.906 respectively; the effect disappeared at gamma=0.01 because the 48-cell spatial quantile equals the maximum at that resolution.",
   "metrics": {
    "baseline": "Max-residual conformal: q=0.0041301 for all gamma; mean width=0.0002891; event rates gamma=.01/.05/.10 = 0.909/0.917/0.930; mean spatial hit fractions = 0.977/0.977/0.977",
    "idea": "Spatial-quantile conformal: q=.0041301/.0039117/.0036424 for gamma=.01/.05/.10; mean widths=.0002891/.0002738/.0002550; event rates=.909/.903/.906; mean spatial hit fractions=.977/.973/.967"
   },
   "how_to_run": "python3 spatial_quantile_conformal.py",
   "files": [
    "spatial_quantile_conformal.py",
    "results.json"
   ],
   "limitations": "This is a small synthetic smooth-operator surrogate rather than a trained FNO on 2-D Darcy flow. Only one fixed train/calibration/test split was run; repeated calibration-size studies, Bonferroni and pointwise baselines, and a nonuniform-weight end-to-end coverage experiment were not performed."
  },
  "status": "mech_ok_no_baseline",
  "status_label": "Mechanism confirmed, baseline not beaten",
  "updated_at": "2026-09-02T12:30:10",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": true,
    "tested": true
   },
   "practical_benchmark": {
    "beats_baseline": false,
    "tested": true,
    "verdict": "no_effect"
   },
   "toy_mechanism_gate": {
    "confirmed": true,
    "tested": true
   }
  }
 }
}