# Conformal Residual Certificates for Neural Rollouts

- ID: 1891
- Canonical URL: https://synthcore.org/idea/1891/conformal-residual-certificates-for-neural-rollouts
- API JSON: https://synthcore.org/api/idea/1891.json
- API Markdown: https://synthcore.org/api/idea/1891.md
- Verification status: failed_benchmark
- Source: [arXiv:2608.06419](https://arxiv.org/abs/2608.06419)
- Category: theory
- Solves: stability, generalization
- ML areas: world-model, ssm, regularization, inference-speedup
- Math tags: statistics, dynamical-systems, control-theory, linear-algebra
- Ratings: usefulness 7/10; difficulty 4/10; novelty 6/10

## Idea description

Attach a finite-sample conformal error radius to a neural surrogate of a dynamical or sequence model, and propagate that radius through the model's local sensitivity. The model should expose a calibrated prediction set or abstain whenever the accumulated bound exceeds a task-specific tolerance, making long-horizon failure a measurable coverage event rather than an unobserved drift.

## Mathematical statement

Let z denote an input window or Fourier-coefficient vector, F_s(z) the unknown true input-output map, and F_hat(z) the trained neural surrogate. On a calibration set of n_cal examples, define nonconformity scores r_i = ||y_i - F_hat(z_i)||_2. For miscoverage level alpha, let q_(1-alpha) be the conformal quantile with the finite-sample rank correction. Then the marginal guarantee is P(||F_s(z) - F_hat(z)||_2 <= q_(1-alpha)) >= 1 - alpha. If a downstream rollout has local Lipschitz sensitivities L_t, a conservative horizon-H radius is R_H = q_(1-alpha) times the sum from t = 0 to H - 1 of the products of later L_j values. When every L_j <= L, this becomes q_(1-alpha)(L^H - 1)/(L - 1), or Hq_(1-alpha) when L = 1. The transferable mechanism is calibrated residual coverage combined with geometric error amplification.

## Key formulas

- r_i = ||y_i - F_hat(z_i)||_2, and q_(1-alpha) = Quantile_rank(r_1,...,r_n_cal), where rank = ceil((n_cal + 1)(1 - alpha))
- P(||F_s(z) - F_hat(z)||_2 <= q_(1-alpha)) >= 1 - alpha
- R_H = q_(1-alpha) * sum_{t=0}^{H-1} product_{j=t+1}^{H-1} L_j <= q_(1-alpha) * (L^H - 1)/(L - 1)
- L_j = ||partial F_hat / partial x evaluated at (x_j,u_j)||_2

## Implementation notes

First, integrate the method after training any one-step or block-prediction model F_hat_theta(x_t,u_t), including an MLP, GRU, Transformer dynamics model, or Fourier-domain sequence model. Reserve temporally separated calibration trajectories and do not use them for gradient updates. Second, implement the following procedure: fit theta on training trajectories; compute r_i = norm(x_(i,t+1) - F_hat_theta(x_(i,t),u_(i,t))) on calibration windows; choose q using the corrected conformal order statistic; during a rollout, estimate each Jacobian spectral norm L_t with one or two power iterations through automatic differentiation; return the prediction and R_H = q times the sum of products of later gains. Trigger abstention, replanning, a shorter rollout, or a fallback model if R_H exceeds a task tolerance epsilon. The paper supplies the residual-quantile and coverage mechanism; the implementation must empirically estimate Jacobian gains, temporal dependence effects, and whether one-step coverage transfers to trajectories. The first cheap experiment should use a small MLP or GRU on a damped pendulum or synthetic Duffing oscillator, with alpha equal to 0.05 or 0.1, compared against an uncertified rollout. The predicted signature is one-step held-out coverage at least 1 - alpha under exchangeable calibration data and a rollout radius following the geometric law R_H proportional to (L^H - 1)/(L - 1). Measure the horizon at which R_H crosses epsilon and test whether the empirical crossing is within 20 percent of the predicted crossing.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: no_effect
- Verdict: Built a reproducible conformal-residual certificate for a learned one-step dynamical surrogate, including corrected finite-sample quantiles, autodiff local gains, radius propagation, and abstention. The mechanism manifested: Monte Carlo one-step coverage was 0.899–0.905 for target 0.90 across calibration sizes, direct and closed-form geometric radii agreed to machine precision, and scalar recurrence crossing horizons matched predictions exactly for L=1.0, 1.1, and 1.3. In the learned rollout, one-step and all-horizon certificate coverage were both 1.0, but the baseline had zero final tolerance violations and the certificate abstained around horizon 10.9, demonstrating conservatism rather than a performance win.

### Mechanism check

- Verdict: Built a reproducible conformal-residual certificate for a learned one-step dynamical surrogate, including corrected finite-sample quantiles, autodiff local gains, radius propagation, and abstention. The mechanism manifested: Monte Carlo one-step coverage was 0.899–0.905 for target 0.90 across calibration sizes, direct and closed-form geometric radii agreed to machine precision, and scalar recurrence crossing horizons matched predictions exactly for L=1.0, 1.1, and 1.3. In the learned rollout, one-step and all-horizon certificate coverage were both 1.0, but the baseline had zero final tolerance violations and the certificate abstained around horizon 10.9, demonstrating conservatism rather than a performance win.
- Confidence: 8/10
- Limitations: The dynamical example is a one-dimensional synthetic stable map rather than a pendulum/Duffing trajectory dataset; calibration/test inputs are i.i.d. rather than temporally dependent trajectories. The scalar crossing sweep uses the certificate recurrence itself, so it validates the formula rather than independently validating distribution-free long-horizon coverage. No FLOP/speed or replanning comparison was tested, and the reported certificate is conservative because local gains are estimated along the predicted path while q is reused at every step.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 1
- Paired wins: 0/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_conformal.py](https://synthcore.org/code/720/bench_conformal.py)
- [bench_report.json](https://synthcore.org/code/720/bench_report.json)
- [report.md](https://synthcore.org/code/720/report.md)
- [report_bench_2026-09-01T011140.md](https://synthcore.org/code/720/report_bench_2026-09-01T011140.md)
- [results.json](https://synthcore.org/code/720/results.json)
- [run_experiment.py](https://synthcore.org/code/720/run_experiment.py)
- [Download all files as ZIP](https://synthcore.org/download/720)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.