# Confidence-Tube Neural Rollouts

- ID: 799
- Canonical URL: https://synthcore.org/idea/799/confidence-tube-neural-rollouts
- API JSON: https://synthcore.org/api/idea/799.json
- API Markdown: https://synthcore.org/api/idea/799.md
- Verification status: beats_baseline
- Source: [arXiv:2607.12343](https://arxiv.org/abs/2607.12343)
- Category: dynamics
- Solves: stability, accuracy, sample-efficiency
- ML areas: world-model, rl, optimizer
- Math tags: control-theory, dynamical-systems, probability, spectral-theory, optimization
- Ratings: usefulness 8/10; difficulty 6/10; novelty 6/10

## Idea description

Augment a learned neural state-space model with an online regularized least-squares confidence set for its local linearization or last-layer dynamics, then propagate a homothetic uncertainty tube around every predicted trajectory. Use the tube to tighten RL action constraints, reject unsafe imagined rollouts, or weight training examples by certified prediction reliability. The mechanism should improve long-horizon behavior specifically when model uncertainty is large, rather than acting as an undifferentiated regularizer.

## Mathematical statement

Assume the true local dynamics are $x_{t+1}=A_*x_t+B_*u_t+w_t$, with bounded disturbance $w_t\in\mathcal W$ and unknown parameter vector $\theta_*=[\operatorname{vec}(A_*)^\top,\operatorname{vec}(B_*)^\top]^\top$. Given regressors $z_t=[x_t^\top,u_t^\top]^\top$ and targets $x_{t+1}$, compute the regularized least-squares estimate $\hat\theta_t=V_t^{-1}\sum_{s<t}z_sx_{s+1}^\top$ with $V_t=\lambda I+\sum_{s<t}z_sz_s^\top$, where $\lambda>0$. Construct a high-probability polytope $\Theta_t=\{\theta:H_t\theta\le h_t\}$ containing $\theta_*$; coordinate-wise intervals are a practical approximation, with half-width $\epsilon_{t,j}$ determined by the non-asymptotic confidence radius and disturbance bound. For nominal feedback rollout $\bar x_{i+1}=(\hat A_t+\hat B_tK)\bar x_i+\hat B_t v_i$, represent the actual state as $x_i=\bar x_i+e_i$. If $e_i\in\alpha_i\mathcal S$ for a template set $\mathcal S$, choose $\alpha_{i+1}$ so every model vertex and disturbance satisfy $e_{i+1}\in\alpha_{i+1}\mathcal S$. If $\gamma_t=\max_{\theta\in\mathcal V(\Theta_t)}\|A_\theta+B_\theta K\|_{\mathcal S}<1$ and $w_{\mathcal S}$ bounds the disturbance, then $\alpha_{i+1}\le\gamma_t\alpha_i+w_{\mathcal S}$ and $\alpha_i\le\gamma_t^i\alpha_0+w_{\mathcal S}(1-\gamma_t^i)/(1-\gamma_t)$.

## Key formulas

- $$V_t=\lambda I+\sum_{s=0}^{t-1}z_sz_s^\top,\qquad \hat\theta_t=V_t^{-1}\sum_{s=0}^{t-1}z_sx_{s+1}^\top,\qquad z_s=\begin{bmatrix}x_s\\u_s\end{bmatrix}.$$
- $$\Theta_t=\{\theta\mid H_t\theta\le h_t\}=\bar\Theta_t^{\mathrm p}\cap\Theta_{t-1},\qquad \bar\Theta_t^{\mathrm p}=\{\theta\mid |\theta_j-\hat\theta_{t,j}|\leq\epsilon_{t,j}\ \forall j\}.$$
- $$\alpha_{i+1}\mathcal S\supseteq\operatorname{co}_{\theta\in\mathcal V(\Theta_t)}\left((A_\theta+B_\theta K)\alpha_i\mathcal S\right)\oplus\mathcal W,\qquad \alpha_{i+1}\leq\gamma_t\alpha_i+w_{\mathcal S},\quad \gamma_t=\max_{\theta\in\mathcal V(\Theta_t)}\|A_\theta+B_\theta K\|_{\mathcal S}<1.$$
- $$\mathcal X_i^{\mathrm{safe}}=\mathcal X\ominus\alpha_i\mathcal S,\qquad \mathcal U_i^{\mathrm{safe}}=\mathcal U\ominus K(\alpha_i\mathcal S).$$

## Implementation notes

First, integrate the method into a neural state-space model $f_\phi(x,u)$ used by MPC or model-based RL. Treat the final linear layer, or a local Jacobian pair $(A,B)$, as the uncertain dynamics object. Around the current nominal state, compute Jacobians with automatic differentiation and maintain RLS statistics from observed transitions. Initialize $V=\lambda I$, $b=0$, and a parameter polytope $\Theta$. For each transition, form $z=[x;u]$, update $V\leftarrow V+zz^\top$ and $b\leftarrow b+zx_+^\top$, calculate $\hat\theta=V^{-1}b$, obtain confidence widths $\epsilon_j$, and intersect the new interval box with the previous $\Theta$. During planning, enumerate box vertices $\theta^{(q)}$, calculate $\gamma=\max_q\|A^{(q)}+B^{(q)}K\|_{\mathcal S}$, propagate $\alpha_{i+1}=\gamma\alpha_i+w_{\mathcal S}$, and optimize only over $\bar x_i\in\mathcal X\ominus\alpha_i\mathcal S$ and $v_i$ such that the feedback input remains in $\mathcal U\ominus K(\alpha_i\mathcal S)$. RLS updates and vertex propagation are computed directly; disturbance bounds and confidence calibration are estimated from residual quantiles with a safety multiplier; support functions of $\mathcal S$ can be evaluated by linear programs. Start with a 2-D linear system plus a small MLP residual, then test CartPole or constrained pendulum. Compare nominal MPC and ensemble-MPC. The predicted signature is one-step residual coverage at the declared confidence level, confidence widths decreasing approximately like $1/\sqrt{\lambda_{\min}(V_t)}$, and tube radii approaching $w_{\mathcal S}/(1-\gamma)$. Sweeping feedback gain should reveal a rollout-divergence boundary near $\gamma=1$.

## Verification

- Status: beats_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: helps
- Verdict: Built a deterministic scalar linear-system MVP implementing RLS confidence widths, interval uncertainty vertices, homothetic tube propagation, and tightened safety filtering. The core predictions manifested: confidence-width scaling had observed log-log exponents 1.000 and 1.000 versus predicted 1; the tube boundary straddled the predicted gamma=1 transition (stable up to gamma=0.9745, unstable from gamma=1.021); and exact vertex rollout matched the scalar recurrence within 8.6e-9. In the uncertainty sweep, nominal filtering accepted all 9 candidates while tube filtering rejected the highest-uncertainty case, which had 3.2% sampled unsafe successors.

### Mechanism check

- Verdict: Built a deterministic scalar linear-system MVP implementing RLS confidence widths, interval uncertainty vertices, homothetic tube propagation, and tightened safety filtering. The core predictions manifested: confidence-width scaling had observed log-log exponents 1.000 and 1.000 versus predicted 1; the tube boundary straddled the predicted gamma=1 transition (stable up to gamma=0.9745, unstable from gamma=1.021); and exact vertex rollout matched the scalar recurrence within 8.6e-9. In the uncertainty sweep, nominal filtering accepted all 9 candidates while tube filtering rejected the highest-uncertainty case, which had 3.2% sampled unsafe successors.
- Confidence: 9/10
- Limitations: This is a scalar/low-dimensional numerical MVP rather than a neural state-space model, MPC optimizer, CartPole, or RL benchmark. Confidence widths use an empirically calibrated multiplier rather than a formal theorem-derived radius, and the safety test uses sampled model uncertainty rather than exhaustive multi-step closed-loop violation statistics.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 8/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/151/bench_report.json)
- [confidence_tube_experiment.py](https://synthcore.org/code/151/confidence_tube_experiment.py)
- [report.md](https://synthcore.org/code/151/report.md)
- [report_bench_2026-08-30T174506.md](https://synthcore.org/code/151/report_bench_2026-08-30T174506.md)
- [results.json](https://synthcore.org/code/151/results.json)
- [stage2_confidence_tube_bench.py](https://synthcore.org/code/151/stage2_confidence_tube_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/151)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.