# Reachset-Conformance Noise Calibration

- ID: 1705
- Canonical URL: https://synthcore.org/idea/1705/reachset-conformance-noise-calibration
- API JSON: https://synthcore.org/api/idea/1705.json
- API Markdown: https://synthcore.org/api/idea/1705.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.02308](https://arxiv.org/abs/2608.02308)
- Category: regularization
- Solves: stability, generalization, accuracy
- ML areas: rnn, ssm, world-model, regularization, training-dynamics
- Math tags: control-theory, dynamical-systems, optimization, probability, linear-algebra
- Ratings: usefulness 7/10; difficulty 5/10; novelty 8/10

## Idea description

Calibrate process and observation uncertainty bounds by requiring a learned neural dynamical model to contain calibration trajectories in its reachable sets, instead of fitting a Gaussian noise model. The resulting bounds can control an uncertainty-aware loss, trigger teacher forcing or re-observation, and identify latent coordinates whose dynamics are not adequately modeled. This transfers the paper's conformance principle into a falsifiable training monitor and adaptive rollout schedule.

## Mathematical statement

Let $\hat f_\theta$ be a learned latent transition map, $\hat h_\theta$ its decoder, and let $x_k$ be an encoded latent state. Define one-step residuals $r^x_k=x_{k+1}-\hat f_\theta(x_k,u_k)$ and observation residuals $r^y_k=y_k-\hat h_\theta(x_k)$. For componentwise bounded uncertainty, choose the smallest nonnegative bounds $\epsilon_x,\epsilon_y$ such that $|r^x_k|\leq\epsilon_x$ and $|r^y_k|\leq\epsilon_y$ for every calibration trajectory, optionally allowing an outlier fraction $\delta$. Equivalently, the conformant bounds are componentwise maxima, $\epsilon_{x,i}=\max_k|r^x_{k,i}|$ and $\epsilon_{y,j}=\max_k|r^y_{k,j}|$, or empirical $(1-\delta)$ quantiles for contaminated data. A rollout is conformant when its reachable set generated with these bounds intersects every observation-consistency set. The mechanism predicts that the first violation rate on held-out data is approximately the calibration outlier rate $\delta$; systematic violation growth indicates model mismatch rather than merely stochastic uncertainty.

## Key formulas

- $$r^x_k=x_{k+1}-\hat f_\theta(x_k,u_k),\qquad r^y_k=y_k-\hat h_\theta(x_k).$$
- $$\epsilon_{x,i}=\max_{k\in\mathcal{D}_{\mathrm{cal}}}|r^x_{k,i}|,\qquad \epsilon_{y,j}=\max_{k\in\mathcal{D}_{\mathrm{cal}}}|r^y_{k,j}|.$$
- $$\mathcal{X}_{k+1|k}=\hat f_\theta(\mathcal{X}_k,u_k)\oplus[-\epsilon_x,\epsilon_x],\qquad \mathcal{C}_{k+1}=\mathcal{X}_{k+1|k}\cap\{x:|\hat h_\theta(x)-y_{k+1}|\leq\epsilon_y\}.$$
- $$\widehat p_{\mathrm{violation}}=\frac{1}{M}\sum_{m=1}^{M}\mathbf{1}\{\mathcal{C}^{(m)}_{k}=\varnothing\},\qquad \widehat p_{\mathrm{violation}}\approx\delta\text{ under calibrated bounds.}$$

## Implementation notes

1. Integration point: add a calibration phase after ordinary teacher-forced training of a GRU, neural ODE discretization, or latent SSM. Store encoded states, transitions, and decoder outputs on a held-out calibration split. 2. Pseudocode: compute `rx=x_next-f_theta(x,u)` and `ry=y-h_theta(x)`; set `eps_x=quantile(abs(rx),1-delta)` and `eps_y=quantile(abs(ry),1-delta)` componentwise; during rollout propagate an interval or zonotope using these bounds; after each observed target, test whether the predicted output interval intersects `[y-eps_y,y+eps_y]`; if not, increase teacher-forcing frequency, reduce rollout length, or apply a conformant-loss penalty `relu(dist(y,h(X))-eps_y)`. Recalibrate periodically as the model changes. 3. Computed directly: residual bounds and the nonempty reachset-consistency test. Estimated empirically: quantiles, latent encoder uncertainty, and the nonlinear interval/Jacobian bound for $f_\theta(\mathcal{X})$. Use a conservative Lipschitz bound or sampled corners for the latter. 4. First cheap experiment: train a GRU and this calibrated variant on noisy sine-wave and Lorenz-63 forecasting with bounded uniform noise, then test on a shifted disturbance amplitude. Sweep $\delta\in\{0,0.01,0.05,0.1\}$. The predicted signature is held-out empty-intersection frequency near $\delta$ on in-distribution data, while it rises sharply under disturbance amplitudes exceeding the calibrated bound. Plot violation rate and forecast RMSE versus rollout horizon; the conformant model should show an early-warning kink at the horizon where reachable output width first exceeds the observation tolerance, rather than silently diverging.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a scalar latent dynamical-system MVP implementing componentwise residual quantile calibration and one-step reachable-set/observation-consistency testing. Held-out latent and observation violation rates matched delta within 0.002, calibrated bounds scaled linearly with process amplitude, and shifted-disturbance reachset violations rose sharply from 0.003 at 1x to 0.489 at 3x. The direct empty-intersection rate is lower than delta in-distribution because latent and observation bounds are Minkowski-added, making that test conservative.

### Mechanism check

- Verdict: Built a scalar latent dynamical-system MVP implementing componentwise residual quantile calibration and one-step reachable-set/observation-consistency testing. Held-out latent and observation violation rates matched delta within 0.002, calibrated bounds scaled linearly with process amplitude, and shifted-disturbance reachset violations rose sharply from 0.003 at 1x to 0.489 at 3x. The direct empty-intersection rate is lower than delta in-distribution because latent and observation bounds are Minkowski-added, making that test conservative.
- Confidence: 9/10
- Limitations: Only a scalar linear identity-decoder system was tested; no GRU, Lorenz-63, nonlinear interval propagation, adaptive teacher forcing, conformant-loss training, multidimensional latent coordinates, or long-horizon forecasting was evaluated. The Gaussian comparison is a simple residual-width baseline rather than a tuned probabilistic model.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/629/bench_report.json)
- [reachset_experiment.py](https://synthcore.org/code/629/reachset_experiment.py)
- [report.md](https://synthcore.org/code/629/report.md)
- [report_bench_2026-08-31T200532.md](https://synthcore.org/code/629/report_bench_2026-08-31T200532.md)
- [results.json](https://synthcore.org/code/629/results.json)
- [stage2_bench.py](https://synthcore.org/code/629/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/629)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.