# Residual-Relaxed Trajectory Sampling

- ID: 543
- Canonical URL: https://synthcore.org/idea/543/residual-relaxed-trajectory-sampling
- API JSON: https://synthcore.org/api/idea/543.json
- API Markdown: https://synthcore.org/api/idea/543.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.06950](https://arxiv.org/abs/2607.06950)
- Category: sampling
- Solves: stability, accuracy, sample-efficiency
- ML areas: diffusion-sampling, world-model, optimizer
- Math tags: probability, dynamical-systems, optimization, control-theory
- Ratings: usefulness 7/10; difficulty 4/10; novelty 6/10

## Idea description

Make a diffusion policy or MPPI-style action-sequence sampler less committed to model-predicted cost rankings when the learned world model is inaccurate. Estimate a normalized prediction residual or ensemble disagreement, increase the sampling temperature with that residual, and retain ordinary low-temperature exploitation when the model is accurate.

## Mathematical statement

The paper models the nominal predictor as $\mathbf{x}_{k+1}=f_{\theta}(\mathbf{x}_k,\mathbf{u}_k)$, while the real transition is $\mathbf{x}_{k+1}=\tilde f(\mathbf{x}_k,\mathbf{u}_k)+\mathbf{w}_k$; $\mathbf{x}_k$ is the state, $\mathbf{u}_k$ the action, $f_\theta$ the learned or nominal dynamics, $\tilde f$ the unknown plant dynamics, and $\mathbf{w}_k$ process disturbance. Define the observed residual $r_k=\|\mathbf{y}_{k+1}-f_\theta(\mathbf{y}_k,\mathbf{u}_k)\|_2/(\sigma_r+\operatorname{EMA}[\|\mathbf{y}_{k+1}-f_\theta(\mathbf{y}_k,\mathbf{u}_k)\|_2])$, where $\mathbf{y}_k=\mathbf{x}_k+\mathbf{v}_k$ is the noisy observation, $\sigma_r>0$ prevents division by zero, and EMA is an exponential moving average. For candidate trajectories with predicted costs $S_i$ and perturbations $\epsilon_i$, use MPPI weights $w_i=\exp(-S_i/\lambda_k)/\sum_j\exp(-S_j/\lambda_k)$ and update the nominal action sequence by $U\leftarrow U+\sum_iw_i\epsilon_i$. The adaptation is $\lambda_k=\lambda_0(1+\alpha\,\operatorname{clip}(r_k,0,r_{\max}))$, with baseline temperature $\lambda_0>0$ and gain $\alpha\ge0$. This implements the paper's residual-over-temperature principle: cost-weight perturbations scale upward with mismatch and scale downward as $\lambda$ increases.

## Key formulas

- $$\mathbf{x}_{k+1}=f_{\theta}(\mathbf{x}_k,\mathbf{u}_k),\qquad \mathbf{x}_{k+1}=\tilde f(\mathbf{x}_k,\mathbf{u}_k)+\mathbf{w}_k,\qquad \mathbf{y}_k=\mathbf{x}_k+\mathbf{v}_k.$$
- $$w_i=\frac{\exp(-S_i/\lambda_k)}{\sum_{j=1}^{M}\exp(-S_j/\lambda_k)},\qquad U\leftarrow U+\sum_{i=1}^{M}w_i\epsilon_i.$$
- $$r_k=\frac{\left\|\mathbf{y}_{k+1}-f_{\theta}(\mathbf{y}_k,\mathbf{u}_k)\right\|_2}{\sigma_r+\operatorname{EMA}\left[\left\|\mathbf{y}_{k+1}-f_{\theta}(\mathbf{y}_k,\mathbf{u}_k)\right\|_2\right]},\qquad \lambda_k=\lambda_0\left(1+\alpha\,\operatorname{clip}(r_k,0,r_{\max})\right).$$

## Implementation notes

Integrate this into the candidate-action sampling stage of a learned world-model controller or diffusion policy sampler. At deployment, maintain a nominal action sequence U of horizon H and a residual EMA. Before the first step, initialize lambda=lambda0. At each control cycle, draw M action perturbations epsilon_i, roll each U+epsilon_i through the world model f_theta, compute trajectory cost S_i, and calculate weights with the temperature lambda_k rather than a fixed temperature. Execute the first action of the weighted sequence, observe the next state y_{k+1}, and compute d_k=||y_{k+1}-f_theta(y_k,u_k)||_2. Update residual_ema <- beta*residual_ema+(1-beta)*d_k and r_k <- d_k/(sigma_r+residual_ema). Then set lambda_{k+1}=lambda0*(1+alpha*clip(r_k,0,rmax)); shift the remaining sequence and repeat. For a diffusion policy, replace the MPPI weighted update by sampling the denoising trajectory or action candidates with logits divided by lambda_k, using the same residual schedule. The math-computed quantities are the residual normalization, temperature, and exponential importance weights; alpha, beta, sigma_r, and rmax are tuned on validation rollouts. A cheap test is a 2D point-mass or CartPole learned-dynamics benchmark with injected dynamics changes at evaluation time. Compare fixed-temperature MPPI or diffusion sampling against this method at equal model-rollout count. Success means lower catastrophic constraint or failure rate under mismatch, flatter performance degradation as mismatch grows, and near-identical nominal reward when residuals remain small.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: inconclusive
- Verdict: Built residual-relaxed MPPI for a 2D point-mass benchmark and numerically verified all three mechanism predictions. Temperature scaling matched the prescribed formula with zero error; effective sample size increased monotonically from 1.31 to 58.78 as temperature increased; and the paired-action update matched -tanh(g/lambda) within 1.1e-16. The mechanism is real, but this small benchmark did not show a control win: residual relaxation slightly worsened mean return at every tested mismatch level while success remained 100% for both methods.

### Mechanism check

- Verdict: Built residual-relaxed MPPI for a 2D point-mass benchmark and numerically verified all three mechanism predictions. Temperature scaling matched the prescribed formula with zero error; effective sample size increased monotonically from 1.31 to 58.78 as temperature increased; and the paired-action update matched -tanh(g/lambda) within 1.1e-16. The mechanism is real, but this small benchmark did not show a control win: residual relaxation slightly worsened mean return at every tested mismatch level while success remained 100% for both methods.
- Confidence: 9/10
- Limitations: Only a small handcrafted 2D point-mass task was tested, with one fixed model and mismatch family, 30 episodes per condition, and no hyperparameter tuning or ensemble-disagreement variant. The benchmark had no failures to reduce, so it cannot establish improved robustness; the observed return signal favored fixed temperature.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.44995
- Paired wins: 4/8
- Benchmark verdict: no significant win

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/252/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/252/bench_report.json)
- [bench_stage2.py](https://synthcore.org/code/252/bench_stage2.py)
- [experiment.py](https://synthcore.org/code/252/experiment.py)
- [report.md](https://synthcore.org/code/252/report.md)
- [report_bench_2026-09-02T214609.md](https://synthcore.org/code/252/report_bench_2026-09-02T214609.md)
- [results.json](https://synthcore.org/code/252/results.json)
- [Download all files as ZIP](https://synthcore.org/download/252)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.