# Conformal Lower-Clearance Certificate for Neural Selectors

- ID: 2747
- Canonical URL: https://synthcore.org/idea/2747/conformal-lower-clearance-certificate-for-neural-selectors
- API JSON: https://synthcore.org/api/idea/2747.json
- API Markdown: https://synthcore.org/api/idea/2747.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.26533](https://arxiv.org/abs/2608.26533)
- Category: theory
- Solves: stability, generalization, accuracy
- ML areas: optimizer, rl, world-model, training-dynamics
- Math tags: probability, statistics, control-theory
- Ratings: usefulness 8/10; difficulty 4/10; novelty 5/10

## Idea description

Attach a finite-sample lower safety certificate to the trajectory selected by a neural planner or policy by calibrating the difference between predicted and realized clearance. A lower-tail CVaR of sampled neural predictions can provide the raw margin, while conformal calibration subtracts an empirical correction that absorbs predictor bias and sampling error.

## Mathematical statement

Let I_i be a planning episode, Y_i the realized minimum signed clearance of the trajectory selected using I_i, and M_i its predicted safety margin. Define the nonconformity score R_i=M_i-Y_i. For miscoverage alpha and n calibration episodes, choose q as the suitably rounded empirical (1-alpha)-quantile of R_i. The deployed certificate is C(I)=M(I)-q. Under exchangeability of calibration and test episodes, the probability that realized clearance is at least the certificate is at least 1-alpha. If Z_1,...,Z_K are sampled predicted clearances and gamma is the lower-tail probability, use the empirical CVaR M_CVaR=max_eta[eta-(gamma K)^(-1) sum_k(eta-Z_k)_+] as M before calibration.

## Key formulas

- $$R_i=M_i-Y_i,\qquad q=\operatorname{Quantile}_{1-\alpha}\{R_1,\ldots,R_n\},\qquad C(I)=M(I)-q.$$
- $$\Pr\left(Y(I_{n+1})\ge M(I_{n+1})-q\right)\ge 1-\alpha\quad\text{under exchangeability}.$$
- $$M_{\mathrm{CVaR},\gamma}(I)=\max_{\eta\in\mathbb{R}}\left[\eta-\frac{1}{\gamma K}\sum_{k=1}^{K}(\eta-Z_k(I))_+\right].$$
- $$q=R_{(k)},\qquad k=\left\lceil(n+1)(1-\alpha)\right\rceil,\qquad R_{(1)}\le\cdots\le R_{(n)}.$$

## Implementation notes

1. Integration point: leave the neural planner and candidate-selection rule unchanged, but make the network output either a nominal minimum-clearance prediction or K stochastic clearance predictions. For each episode, compute M for the trajectory actually selected by the planner and record its realized minimum signed OBB clearance after rollout. 2. Pseudocode: collect n calibration sessions; compute M_i and Y_i; set R_i=M_i-Y_i; sort the residuals; select q=R_(ceil((n+1)(1-alpha))); during deployment return C=M_new-q. Reject, replan, or reduce speed when C<0. If using samples, first optimize the one-dimensional CVaR expression over eta. 3. Computed from the mechanism: residual definition, order-statistic quantile, and CVaR formula. Estimated empirically: residual distribution, prediction samples, and validity of exchangeability. Selection must occur before R_i is computed, otherwise the certificate does not cover the selected output. 4. First experiment: use a frozen nuPlan planner or a small neural trajectory scorer on 300 logged sessions, with alpha=0.1, comparing nominal and CVaR margins. The predicted signature is held-out coverage P(Y>=C)>=0.9, typically near 0.90 to 0.97, while increasing alpha to 0.2 should reduce q and move coverage toward 0.8. Report q, certificate nonnegativity, and coverage confidence intervals. Deliberately shift test scenes to expose the expected failure under distribution shift.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a synthetic conformal lower-clearance certificate with empirical lower-tail CVaR and finite-sample order-statistic calibration. The CVaR optimizer matched the direct lower-tail computation to 2.2e-16; under exchangeability, coverage tracked the predicted targets (alpha=.05/.1/.2/.3: .950/.901/.804/.702), while uncalibrated CVaR coverage at alpha=.1 was only .771. Smaller gamma monotonically reduced the calibrated residual correction (q=-.009, .025, .072, .164 for gamma=.1,.25,.5,1), and the deliberate test shift reduced coverage to .734, demonstrating the expected exchangeability limitation.

### Mechanism check

- Verdict: Built a synthetic conformal lower-clearance certificate with empirical lower-tail CVaR and finite-sample order-statistic calibration. The CVaR optimizer matched the direct lower-tail computation to 2.2e-16; under exchangeability, coverage tracked the predicted targets (alpha=.05/.1/.2/.3: .950/.901/.804/.702), while uncalibrated CVaR coverage at alpha=.1 was only .771. Smaller gamma monotonically reduced the calibrated residual correction (q=-.009, .025, .072, .164 for gamma=.1,.25,.5,1), and the deliberate test shift reduced coverage to .734, demonstrating the expected exchangeability limitation.
- Confidence: 9/10
- Limitations: This is a synthetic selected-trajectory simulator, not a nuPlan or real neural-planner integration; no learned model, OBB geometry, planner replanning, confidence intervals, or GPU benchmark was tested. Distribution-shift failure means the formal guarantee should not be applied outside the calibration distribution.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/1058/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/1058/bench_report.json)
- [certificate_experiment.py](https://synthcore.org/code/1058/certificate_experiment.py)
- [report.md](https://synthcore.org/code/1058/report.md)
- [report_bench_2026-09-01T232458.md](https://synthcore.org/code/1058/report_bench_2026-09-01T232458.md)
- [results.json](https://synthcore.org/code/1058/results.json)
- [Download all files as ZIP](https://synthcore.org/download/1058)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.