✓✓ Beats tuned baseline 2026

Conformal Residual Gate for Latent Filtering

Implementation & benchmark of arXiv:2607.14273 — Learning reduced-order latent linear models for Kalman filtering of nonlinear systems

Usefulness7/10

Difficulty5/10

Novelty7/10

Regularization Accuracy Stability Sample efficiency Ssm Rnn World model Training dynamics Statistics Probability Control theory Dynamical systems

Source paper: Learning reduced-order latent linear models for Kalman filtering of nonlinear systems arXiv:2607.14273 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use held-out calibration trajectories to convert decoded latent-filter residuals into distribution-free error thresholds. At inference, the threshold can flag unreliable estimates, inflate measurement uncertainty, request an additional observation, or switch to a higher-dimensional fallback model instead of silently propagating a bad latent state.

Formulas

$$s_k=\|x_k-D_\phi(\hat z_k)\|_2\quad\text{or}\quad s_k=\left\|\operatorname{diag}(S_k)^{-1/2}\bigl(x_k-D_\phi(\hat z_k)\bigr)\right\|_\infty.$$

$$q=s_{(\lceil(m+1)(1-\alpha)\rceil)},\qquad \mathcal{C}_k=\{x:s(x,\hat x_k)\leq q\},\qquad \Pr\{x_k\in\mathcal{C}_k\}\gtrsim1-\alpha.$$

$$S_k=J_kP_kJ_k^\top+\Sigma_D,\qquad J_k=\frac{\partial D_\phi}{\partial z}(\hat z_k).$$

Mathematical statement

Let $x_k$ be the true state and $\hat{x}_k=D_\phi(\hat z_k)$ the filtered estimate. Define a nonconformity score, either an unnormalized residual $s_k=\|x_k-\hat{x}_k\|_2$ or a covariance-normalized residual. On a calibration set with $m$ exchangeable scores, sort them as $s_{(1)}\leq\cdots\leq s_{(m)}$ and choose $q=s_{(\lceil(m+1)(1-\alpha)\rceil)}$, where $\alpha$ is the target miscoverage. The resulting set has marginal coverage approximately at least $1-\alpha$ under exchangeability. For uncertainty normalization, $S_k=J_kP_kJ_k^\top+\Sigma_D$, where $J_k=\partial D_\phi/\partial z$, $P_k$ is the latent Kalman covariance, and $\Sigma_D$ is decoder residual covariance.

Implementation notes

Train the latent Kalman model on one set of trajectories and reserve complete trajectories with ground-truth states for calibration. Freeze the model, run the filter on the calibration set, compute one nonconformity score per time step, sort the scores, and store the finite-sample quantile $q$ using index $\lceil(m+1)(1-\alpha)\rceil$. At deployment the true state is unavailable, so use the measurement innovation $e_k=y_k-h_\omega(\bar z_k,u_k)$, its Mahalanobis norm, or the predicted normalized uncertainty as a trigger surrogate. If the trigger exceeds the calibrated threshold, apply a concrete intervention: inflate $R\leftarrow\beta R$, inflate $P_k\leftarrow\beta P_k$, request another measurement, or switch to a fallback model with latent dimension $2r$. Calibrate thresholds separately by forecast horizon if residuals drift over time. The paper contributes the distribution-free calibration mechanism; the trigger and intervention are engineering adaptations. Test on a noisy heat-diffusion benchmark or nonlinear oscillator against an ungated latent Kalman filter using $\alpha=0.1$. The prediction is approximately 90 percent held-out coverage under exchangeability, with violations consistent with finite-sample error. Innovation threshold crossings should precede large decoded-state residuals, and covariance inflation should reduce residuals within 5 to 20 steps.

Verification

Beats tuned baseline

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significant win ✓

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

How to read this: The implementation beat the baseline, but the claimed mechanism did not reproduce at benchmark scale. The gain may come from another effect; treat the cause as unresolved.

Stage 1 — Mechanism check agent confidence 9/10

Built a scalar Kalman-filter MVP with held-out conformal residual calibration and an innovation-triggered R-inflation gate. At alpha values 0.05, 0.10, 0.20, and 0.30, observed coverage was 0.963, 0.911, 0.804, and 0.696 versus targets 0.95, 0.90, 0.80, and 0.70. Quantiles decreased monotonically with alpha, calibration quantile standard deviation scaled with fitted exponent -0.537 versus predicted -0.5, and gating reduced contaminated RMSE from 0.3842 to 0.3453, a 10.1% improvement.

Agent confidence: 9/10

Baseline: Contaminated-observation RMSE 0.3842 +/- 0.0932 (ungated Kalman filter)
Idea: Contaminated-observation RMSE 0.3453 +/- 0.087?; 10.13% mean improvement, calibrated innovation threshold 1.634, trigger rate 15.38%; conformal coverage at alpha=0.10 was 0.9115

Limitations: This is a scalar random-walk toy rather than a learned latent decoder or heat-diffusion/nonlinear benchmark. It tests marginal exchangeable coverage and robust measurement gating, not conditional coverage, horizon-specific drift, fallback models, additional observations, GPU performance, or a full latent covariance-normalized decoder residual.

How to run: python3 conformal_gate_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.14273) Sep 2, 2026

Beats the tuned baseline by -0.0002 metric points (p=0.0081, wins 8 / 8 paired seeds; metric: lower is better).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0
Idea mean: 0
Effect (Δ): -0.0002 (n/a; negative = idea better)
Wins: 8 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Not confirmed ✗
Practical effect: Helps
Baseline tuning: swept over 3 configs

Limitations:

The fixed dynamics benchmark exposes supervised rollout prediction rather than a latent Kalman filtering interface, so the implemented intervention is conformal-calibrated residual downweighting during training rather than inference-time measurement gating. Deployment coverage, conditional coverage, horizon-specific thresholds, covariance-normalized decoder residuals, outlier interventions, and fallback models were not tested.

How to run: python3 conformal_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Worked ✓

Agent confidence: 8/10

On the structurally matched dynamics track, the conformal-calibrated residual downweighting system used the same rnn_small architecture and improved mean test MSE from 0.0006132684 to 0.0004085010 across 8 paired seeds. All 8 seeds improved and the paired permutation p-value was 0.0081, so the benchmark verdict is idea better (significant). Coverage confirmation was not obtained because deployment marginal coverage was not directly evaluated.

Baseline: mean test MSE 0.0006132684 +/- 0.0001242455; best lr=0.006
Idea: mean test MSE 0.0004085010 +/- 0.0001104111; best lr=0.006; paired delta=-0.0002047674; p=0.0081; 8/8 wins

Benchmark result