△ Mechanism confirmed, baseline not beaten 2026

Residual-Relaxed Trajectory Sampling

Implementation & benchmark of arXiv:2607.06950 — Residual-Conservative Model Predictive Path Integral Control

Usefulness7/10

Difficulty4/10

Novelty6/10

Sampling Stability Accuracy Sample efficiency Diffusion sampling World model Optimizer Probability Dynamical systems Optimization Control theory

Source paper: Residual-Conservative Model Predictive Path Integral Control arXiv:2607.06950 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Make a diffusion policy or MPPI-style action-sequence sampler less committed to model-predicted cost rankings when the learned world model is inaccurate. Estimate a normalized prediction residual or ensemble disagreement, increase the sampling temperature with that residual, and retain ordinary low-temperature exploitation when the model is accurate.

Formulas

$$\mathbf{x}_{k+1}=f_{\theta}(\mathbf{x}_k,\mathbf{u}_k),\qquad \mathbf{x}_{k+1}=\tilde f(\mathbf{x}_k,\mathbf{u}_k)+\mathbf{w}_k,\qquad \mathbf{y}_k=\mathbf{x}_k+\mathbf{v}_k.$$

$$w_i=\frac{\exp(-S_i/\lambda_k)}{\sum_{j=1}^{M}\exp(-S_j/\lambda_k)},\qquad U\leftarrow U+\sum_{i=1}^{M}w_i\epsilon_i.$$

$$r_k=\frac{\left\|\mathbf{y}_{k+1}-f_{\theta}(\mathbf{y}_k,\mathbf{u}_k)\right\|_2}{\sigma_r+\operatorname{EMA}\left[\left\|\mathbf{y}_{k+1}-f_{\theta}(\mathbf{y}_k,\mathbf{u}_k)\right\|_2\right]},\qquad \lambda_k=\lambda_0\left(1+\alpha\,\operatorname{clip}(r_k,0,r_{\max})\right).$$

Mathematical statement

The paper models the nominal predictor as $\mathbf{x}_{k+1}=f_{\theta}(\mathbf{x}_k,\mathbf{u}_k)$, while the real transition is $\mathbf{x}_{k+1}=\tilde f(\mathbf{x}_k,\mathbf{u}_k)+\mathbf{w}_k$; $\mathbf{x}_k$ is the state, $\mathbf{u}_k$ the action, $f_\theta$ the learned or nominal dynamics, $\tilde f$ the unknown plant dynamics, and $\mathbf{w}_k$ process disturbance. Define the observed residual $r_k=\|\mathbf{y}_{k+1}-f_\theta(\mathbf{y}_k,\mathbf{u}_k)\|_2/(\sigma_r+\operatorname{EMA}[\|\mathbf{y}_{k+1}-f_\theta(\mathbf{y}_k,\mathbf{u}_k)\|_2])$, where $\mathbf{y}_k=\mathbf{x}_k+\mathbf{v}_k$ is the noisy observation, $\sigma_r>0$ prevents division by zero, and EMA is an exponential moving average. For candidate trajectories with predicted costs $S_i$ and perturbations $\epsilon_i$, use MPPI weights $w_i=\exp(-S_i/\lambda_k)/\sum_j\exp(-S_j/\lambda_k)$ and update the nominal action sequence by $U\leftarrow U+\sum_iw_i\epsilon_i$. The adaptation is $\lambda_k=\lambda_0(1+\alpha\,\operatorname{clip}(r_k,0,r_{\max}))$, with baseline temperature $\lambda_0>0$ and gain $\alpha\ge0$. This implements the paper's residual-over-temperature principle: cost-weight perturbations scale upward with mismatch and scale downward as $\lambda$ increases.

Implementation notes

Integrate this into the candidate-action sampling stage of a learned world-model controller or diffusion policy sampler. At deployment, maintain a nominal action sequence U of horizon H and a residual EMA. Before the first step, initialize lambda=lambda0. At each control cycle, draw M action perturbations epsilon_i, roll each U+epsilon_i through the world model f_theta, compute trajectory cost S_i, and calculate weights with the temperature lambda_k rather than a fixed temperature. Execute the first action of the weighted sequence, observe the next state y_{k+1}, and compute d_k=||y_{k+1}-f_theta(y_k,u_k)||_2. Update residual_ema <- beta*residual_ema+(1-beta)*d_k and r_k <- d_k/(sigma_r+residual_ema). Then set lambda_{k+1}=lambda0*(1+alpha*clip(r_k,0,rmax)); shift the remaining sequence and repeat. For a diffusion policy, replace the MPPI weighted update by sampling the denoising trajectory or action candidates with logits divided by lambda_k, using the same residual schedule. The math-computed quantities are the residual normalization, temperature, and exponential importance weights; alpha, beta, sigma_r, and rmax are tuned on validation rollouts. A cheap test is a 2D point-mass or CartPole learned-dynamics benchmark with injected dynamics changes at evaluation time. Compare fixed-temperature MPPI or diffusion sampling against this method at equal model-rollout count. Success means lower catastrophic constraint or failure rate under mismatch, flatter performance degradation as mismatch grows, and near-identical nominal reward when residuals remain small.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built residual-relaxed MPPI for a 2D point-mass benchmark and numerically verified all three mechanism predictions. Temperature scaling matched the prescribed formula with zero error; effective sample size increased monotonically from 1.31 to 58.78 as temperature increased; and the paired-action update matched -tanh(g/lambda) within 1.1e-16. The mechanism is real, but this small benchmark did not show a control win: residual relaxation slightly worsened mean return at every tested mismatch level while success remained 100% for both methods.

Agent confidence: 9/10

Baseline: Mean return at mismatch 0.0/0.35/0.7: -4.319/-3.961/-3.745; success 1.00 at all levels.
Idea: Mean return at mismatch 0.0/0.35/0.7: -4.513/-4.216/-3.964; success 1.00 at all levels. Mechanism checks: exact scaling error 0, ESS 1.31 to 58.78, update-formula error 1.1e-16.

Limitations: Only a small handcrafted 2D point-mass task was tested, with one fixed model and mismatch family, 30 episodes per condition, and no hyperparameter tuning or ensemble-disagreement variant. The benchmark had no failures to reduce, so it cannot establish improved robustness; the observed return signal favored fixed temperature.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.06950) Sep 2, 2026

Ahead of the baseline (−3.9%) but not statistically significant (p=0.45, wins 4 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0006
Idea mean: 0.0005
Effect (Δ): -0 (−3.9%; negative = idea better)
Wins: 4 / 8 paired seeds
p-value: 0.45 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

Only the built-in dynamics track was tested, using 8 epochs, 1200 training samples, 400 test samples, three shared learning-rate values, and three candidate perturbations. This implementation is a supervised training-loop surrogate for residual-adaptive candidate sampling rather than a deployed learned-model MPPI controller; ensemble disagreement, longer horizons, and other tracks were not tested.

How to run: python3 bench_stage2.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 2, 2026 latest Failed ✗

Agent confidence: 9/10

Built and evaluated Residual-Relaxed Trajectory Sampling on the structurally matched dynamics track with shared rnn_small systems, paired seeds, and equal-budget learning-rate sweeps. The trained-model mechanism signature was confirmed because high-residual cases had flatter candidate weights (ESS 28.1731 versus 25.5943). The task-metric improvement was not significant (delta_mean=-2.2066e-05, p=0.44995), so this is not a benchmark win.

Baseline: Test MSE mean 0.0005640739 across 8 seeds; best lr=0.006.
Idea: Test MSE mean 0.0005420081 across 8 seeds; best lr=0.006, alpha=2.0, beta=0.9, sigma=0.05; paired delta=-2.2066e-05, p=0.44995, 4/8 wins.
Mechanism signature: Residual mean 0.0195247, p90 0.0324388; ESS low-residual 25.5943, high-residual 28.1731, ratio 1.10076; confirmed=true.

Benchmark result