Confidence-Tightened Neural Model Predictive Control
Implementation & benchmark of arXiv:2607.28123 — Projection-Regularized Indirect Data-Driven Predictive Control
Source paper: Projection-Regularized Indirect Data-Driven Predictive Control arXiv:2607.28123 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Use a neural dynamics model together with an online uncertainty radius to tighten rollout constraints, action bounds, or latent-state trust regions. The controller or training loop becomes conservative when the predictor is data-poor or exposed to correlated trajectories, and relaxes constraints as uncertainty shrinks. This directly transfers the paper's uniform-in-time confidence-bound and robust recursive-feasibility mechanism to neural world models and safe reinforcement learning.
Formulas
Mathematical statement
Let $\hat f_\theta(z,a)$ predict the next latent state from state $z$ and action $a$, and let $e_t=z_{t+1}-\hat f_\theta(z_t,a_t)$ be the observed residual. Let $\phi_t$ be a feature vector, $\lambda>0$ a ridge parameter, and $V_t=\lambda I+\sum_{s=1}^t\phi_s\phi_s^\top$. A self-normalized martingale-style confidence radius is $r_t(\delta)=\sigma\sqrt{2\log(\det(V_t)^{1/2}/(\det(\lambda I)^{1/2}\delta))}$, where $\sigma$ is a sub-Gaussian noise scale and $\delta$ is the failure probability. If the model Jacobian satisfies $\|\partial f/\partial z\|\le L_f$, then an $H$-step state uncertainty bound is $R_H=r_t(\delta)\sum_{j=0}^{H-1}L_f^j$. For a constraint $c(z,a)\le0$ with Lipschitz constant $L_c$, enforce $c(\hat z_h,\hat a_h)+L_cR_h\le0$ at every predicted horizon step.
Implementation notes
Integrate this into a model-predictive-control loop using a neural dynamics model, or into an offline RL rollout filter. Train an ensemble of small MLPs or a probabilistic recurrent model, but use held-out one-step residuals to estimate $\sigma$ rather than treating ensemble spread as a theorem. At each environment step, compute features $\phi_t$ from the model's final hidden layer, update $V_t$ and the residual covariance, and calculate $r_t(\delta)$. Roll out the nominal action sequence for horizon $H$; propagate uncertainty with the measured or conservatively estimated Jacobian norm $L_f$; then replace every constraint $c\le0$ by $c+L_cR_h\le0$. If no feasible sequence exists, shorten the horizon or use a backup controller. For a differentiable training version, add $\kappa\max(0,c+L_cR_h)^2$ to the rollout loss, with $\kappa$ increased only when empirical coverage falls below $1-\delta$. The paper supplies the uniform-confidence and tightening logic; neural residuals, feature choices, and Jacobian estimates are empirical. First test on CartPole or a two-dimensional point-mass system with noisy dynamics, comparing nominal neural MPC, ensemble MPC, and tightened MPC using identical models and action optimizers. The predicted signature is monotone constraint tightening proportional to $r_t\sum_{j=0}^{H-1}L_f^j$, coverage approximately at least $1-\delta$ on held-out correlated trajectories, and a sharp feasibility collapse as $L_f$ approaches one for long horizons. Report whether empirical violations remain below $\delta$ and whether measured uncertainty growth follows the geometric law within 20%.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Confirmed ✓
Stage 2 · Practical benchmark result: No measurable effect
Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works
Stage 1 — Mechanism check agent confidence 8/10
Built a deterministic confidence-tightened learned-model MPC toy in confidence_tightened_mpc.py and verified the mechanism before comparing controllers. The geometric recurrence matched the closed form to 7.1e-15, uncertainty was exactly linear in sigma, and the horizon-50 sums for L=0.99, 1.0, and 1.01 were 39.50, 50.00, and 64.46. In the matched-model MPC comparison, tightening reduced violations from 1.42% to 0%, but increased mean episode cost from 1.01 to 43.04 because the confidence radius was highly conservative; the safety effect is real, but this is not a practical performance win yet.
- Agent confidence
- 8/10
- Baseline
- Nominal learned-model MPC: violation_rate=0.01419, mean_episode_cost=1.01169, mean_abs_action=0.10041
- Idea
- Confidence-tightened MPC: violation_rate=0.00000, mean_episode_cost=43.04285, mean_abs_action=0.03443; R_H8 was 7.3625, 14.7251, and 29.4501 for sigma scales 0.5, 1, and 2.
Limitations: The learned dynamics surrogate is a small ridge model rather than a neural MLP, and the experiment is a one-dimensional toy system. No formal martingale coverage study on genuinely correlated trajectories, ensemble baseline, adaptive calibration, or larger neural world model was evaluated.
How to run: python3 confidence_tightened_mpc.py
Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.28123)
No significant difference vs. the tuned baseline (−-0.0%, p=1, wins 0 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 0.0002
- Idea mean
- 0.0002
- Effect (Δ)
- 0 (−-0.0%; negative = idea better)
- Wins
- 0 / 8 paired seeds
- p-value
- 1 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- n/a
- Mechanism
- Confirmed ✓
- Practical effect
- No effect
- Baseline tuning
- swept over 3 configs
Only the built-in dynamics track was tested. The intervention used a scalar confidence penalty rather than full online MPC constraint tightening, and no closed-loop action-violation metric was available. The confidence radius was substantially overconservative.
How to run: python3 stage2_confidence_bench.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Stage 2 — Benchmark latest Failed ✗
Agent confidence: 10/10
Implemented and ran the paired dynamics benchmark with matched rnn_small systems and a shared learning-rate sweep. The confidence method achieved 100% empirical H=8 coverage, but the predicted radius was 40.26 versus observed absolute error 0.00927; baseline and idea MSE were identical, with paired delta 0 and permutation p=1.0.
- Baseline
- Best lr=0.01; test MSE mean=0.0001709669, std=0.0001012688.
- Idea
- Best lr=0.01; test MSE mean=0.0001709669, paired delta=0, permutation p=1.0.
Benchmark result
No significant difference vs. the tuned baseline (−-0.0%, p=1, wins 0 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 0.0002
- Idea mean
- 0.0002
- Effect (Δ)
- 0 (−-0.0%; negative = idea better)
- Wins
- 0 / 8 paired seeds
- p-value
- 1 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- n/a
- Mechanism
- Confirmed ✓
- Practical effect
- No effect
- Baseline tuning
- swept over 3 configs
Only the built-in dynamics track was tested. The intervention used a scalar confidence penalty rather than full online MPC constraint tightening, and no closed-loop action-violation metric was available. The confidence radius was substantially overconservative.
How to run: python3 stage2_confidence_bench.py
Artifacts
- bench_report.json 4.5 KB View Raw
- confidence_tightened_mpc.py 6.3 KB View Raw
- report.md 1.5 KB View
- report_bench_2026-08-31T110253.md 2.4 KB View
- results.json 2.4 KB View Raw
- stage2_confidence_bench.py 6.0 KB View Raw