{
 "artifacts": [
  {
   "name": "bench_experiment.py",
   "url": "https://synthcore.org/code/1058/bench_experiment.py"
  },
  {
   "name": "bench_report.json",
   "url": "https://synthcore.org/code/1058/bench_report.json"
  },
  {
   "name": "certificate_experiment.py",
   "url": "https://synthcore.org/code/1058/certificate_experiment.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1058/report.md"
  },
  {
   "name": "report_bench_2026-09-01T232458.md",
   "url": "https://synthcore.org/code/1058/report_bench_2026-09-01T232458.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/1058/results.json"
  }
 ],
 "category": "theory",
 "description": "Attach a finite-sample lower safety certificate to the trajectory selected by a neural planner or policy by calibrating the difference between predicted and realized clearance. A lower-tail CVaR of sampled neural predictions can provide the raw margin, while conformal calibration subtracts an empirical correction that absorbs predictor bias and sampling error.",
 "download_zip": "https://synthcore.org/download/1058",
 "formulas_latex": [
  "$$R_i=M_i-Y_i,\\qquad q=\\operatorname{Quantile}_{1-\\alpha}\\{R_1,\\ldots,R_n\\},\\qquad C(I)=M(I)-q.$$",
  "$$\\Pr\\left(Y(I_{n+1})\\ge M(I_{n+1})-q\\right)\\ge 1-\\alpha\\quad\\text{under exchangeability}.$$",
  "$$M_{\\mathrm{CVaR},\\gamma}(I)=\\max_{\\eta\\in\\mathbb{R}}\\left[\\eta-\\frac{1}{\\gamma K}\\sum_{k=1}^{K}(\\eta-Z_k(I))_+\\right].$$",
  "$$q=R_{(k)},\\qquad k=\\left\\lceil(n+1)(1-\\alpha)\\right\\rceil,\\qquad R_{(1)}\\le\\cdots\\le R_{(n)}.$$"
 ],
 "id": 2747,
 "implementation": "1. Integration point: leave the neural planner and candidate-selection rule unchanged, but make the network output either a nominal minimum-clearance prediction or K stochastic clearance predictions. For each episode, compute M for the trajectory actually selected by the planner and record its realized minimum signed OBB clearance after rollout. 2. Pseudocode: collect n calibration sessions; compute M_i and Y_i; set R_i=M_i-Y_i; sort the residuals; select q=R_(ceil((n+1)(1-alpha))); during deployment return C=M_new-q. Reject, replan, or reduce speed when C\u003c0. If using samples, first optimize the one-dimensional CVaR expression over eta. 3. Computed from the mechanism: residual definition, order-statistic quantile, and CVaR formula. Estimated empirically: residual distribution, prediction samples, and validity of exchangeability. Selection must occur before R_i is computed, otherwise the certificate does not cover the selected output. 4. First experiment: use a frozen nuPlan planner or a small neural trajectory scorer on 300 logged sessions, with alpha=0.1, comparing nominal and CVaR margins. The predicted signature is held-out coverage P(Y\u003e=C)\u003e=0.9, typically near 0.90 to 0.97, while increasing alpha to 0.2 should reduce q and move coverage toward 0.8. Report q, certificate nonnegativity, and coverage confidence intervals. Deliberately shift test scenes to expose the expected failure under distribution shift.",
 "math_summary": "Let I_i be a planning episode, Y_i the realized minimum signed clearance of the trajectory selected using I_i, and M_i its predicted safety margin. Define the nonconformity score R_i=M_i-Y_i. For miscoverage alpha and n calibration episodes, choose q as the suitably rounded empirical (1-alpha)-quantile of R_i. The deployed certificate is C(I)=M(I)-q. Under exchangeability of calibration and test episodes, the probability that realized clearance is at least the certificate is at least 1-alpha. If Z_1,...,Z_K are sampled predicted clearances and gamma is the lower-tail probability, use the empirical CVaR M_CVaR=max_eta[eta-(gamma K)^(-1) sum_k(eta-Z_k)_+] as M before calibration.",
 "math_tags": [
  "probability",
  "statistics",
  "control-theory"
 ],
 "ml_areas": [
  "optimizer",
  "rl",
  "world-model",
  "training-dynamics"
 ],
 "paper": {
  "arxiv_id": "2608.26533",
  "arxiv_url": "https://arxiv.org/abs/2608.26533",
  "summary_what_math_gives_to_ml": "The paper provides two transferable mechanisms for neural trajectory systems: a differentiable separating-axis barrier margin that lower-bounds exact oriented-box clearance, and split conformal calibration that converts an imperfect predicted margin into a statistically valid lower safety certificate. The barrier can be used as a training loss or inference-time rejection criterion for trajectory generators, scorers, world models, and reinforcement-learning policies. The conformal procedure can calibrate the margin of the actually selected neural trajectory without requiring the predictor itself to be statistically correct. Both mechanisms produce falsifiable signatures: the geometric barrier should conservatively track exact collision clearance, while held-out coverage should approach or exceed the target level under exchangeable episodes.",
  "title": "Barrier Function Conformal Safety Clearance Certification with CVaR for Driving Trajectory Selection",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 5,
  "usefulness": 8
 },
 "solves": [
  "stability",
  "generalization",
  "accuracy"
 ],
 "title": "Conformal Lower-Clearance Certificate for Neural Selectors",
 "url": "https://synthcore.org/idea/2747/conformal-lower-clearance-certificate-for-neural-selectors",
 "verification": {
  "benchmark": {
   "track": "dynamics",
   "model": "rnn_small",
   "n_seeds": 8,
   "metric_direction": "lower is better",
   "comparison": {
    "baseline_mean": 0.005887067440198734,
    "idea_mean": 0.011248106741367055,
    "delta_mean": 0.005361039301168321,
    "per_seed_diffs": [
     0.014157527320510916,
     0.0027447003006562665,
     0.002294837417221282,
     0.0029706899097046227,
     0.0020789767940394997,
     0.003361709165230953,
     0.006403836017246542,
     0.008876037484736483
    ],
    "idea_wins": 0,
    "n_pairs": 8,
    "p_value": 0.0081,
    "mde": 0.0035692628707567073,
    "mde_rel_pct": 60.628876890124715,
    "verdict": "idea worse (significant)",
    "system_worked": false
   },
   "baseline": {
    "best_cfg": {
     "lr": 0.006
    },
    "sweep": [
     {
      "cfg": {
       "lr": 0.001
      },
      "mean": 0.01659585232846439
     },
     {
      "cfg": {
       "lr": 0.003
      },
      "mean": 0.012405469547957182
     },
     {
      "cfg": {
       "lr": 0.006
      },
      "mean": 0.005767271446529776
     }
    ],
    "full": {
     "mean": 0.005887067440198734,
     "std": 0.0028623089080256525,
     "per_seed": [
      0.008059674873948097,
      0.004038976971060038,
      0.0031182977836579084,
      0.00785213615745306,
      0.003916425164788961,
      0.0028475807048380375,
      0.005657579749822617,
      0.011605868116021156
     ],
     "n": 8
    }
   },
   "idea": {
    "mean": 0.011248106741367055,
    "std": 0.006258844122725869,
    "per_seed": [
     0.022217202194459013,
     0.006783677271716304,
     0.00541313520087919,
     0.010822826067157683,
     0.005995401958828461,
     0.006209289870068991,
     0.012061415767069159,
     0.02048190560075764
    ],
    "n": 8
   },
   "mechanism_signature": {
    "K": 8,
    "alpha": 0.1,
    "certificate_nonnegative_mean": 0.92,
    "confirmed": true,
    "coverage_mean": 0.9009375,
    "coverage_per_seed": [
     0.92,
     0.9175,
     0.89,
     0.8525,
     0.88,
     0.9225,
     0.9075,
     0.9175
    ],
    "definition": "trained RNN predicted-vs-observed clearance coverage",
    "gamma": 0.25,
    "idea_sweep": [
     {
      "cfg": {
       "lr": 0.001
      },
      "mean": 0.03188629204078374
     },
     {
      "cfg": {
       "lr": 0.003
      },
      "mean": 0.023879009760325764
     },
     {
      "cfg": {
       "lr": 0.006
      },
      "mean": 0.011309210183553048
     }
    ],
    "math_check": {
     "cvar_max_abs_error": 1.1102230246251565e-16,
     "q_is_order_statistic": true,
     "q_rank": 29
    },
    "q_mean": 0.05177069455385208,
    "target_coverage": 0.9
   },
   "custom_track": null
  },
  "mechanism_confirmed": true,
  "peer_reviewed": false,
  "practical_verdict": "harms",
  "stage1_mechanism_check": {
   "worked": true,
   "confidence": 9,
   "verdict": "Built a synthetic conformal lower-clearance certificate with empirical lower-tail CVaR and finite-sample order-statistic calibration. The CVaR optimizer matched the direct lower-tail computation to 2.2e-16; under exchangeability, coverage tracked the predicted targets (alpha=.05/.1/.2/.3: .950/.901/.804/.702), while uncalibrated CVaR coverage at alpha=.1 was only .771. Smaller gamma monotonically reduced the calibrated residual correction (q=-.009, .025, .072, .164 for gamma=.1,.25,.5,1), and the deliberate test shift reduced coverage to .734, demonstrating the expected exchangeability limitation.",
   "metrics": {
    "baseline": "Uncalibrated CVaR certificate C=M: coverage 0.771 at alpha=0.1; under shifted test distribution 0.554.",
    "idea": "Calibrated CVaR: coverage 0.901 at alpha=0.1 with q=0.0249 and certificate nonnegativity 0.9997; shifted coverage 0.734. CVaR identity max error 2.2e-16. Alpha sweep coverage: 0.950, 0.901, 0.804, 0.702 for alpha 0.05, 0.1, 0.2, 0.3."
   },
   "how_to_run": "python3 certificate_experiment.py",
   "files": [
    "certificate_experiment.py",
    "results.json"
   ],
   "limitations": "This is a synthetic selected-trajectory simulator, not a nuPlan or real neural-planner integration; no learned model, OBB geometry, planner replanning, confidence intervals, or GPU benchmark was tested. Distribution-shift failure means the formal guarantee should not be applied outside the calibration distribution."
  },
  "status": "mech_ok_no_baseline",
  "status_label": "Mechanism confirmed, baseline not beaten",
  "updated_at": "2026-09-02T12:30:10",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": true,
    "tested": true
   },
   "practical_benchmark": {
    "beats_baseline": false,
    "tested": true,
    "verdict": "harms"
   },
   "toy_mechanism_gate": {
    "confirmed": true,
    "tested": true
   }
  }
 }
}