# Decision-Weighted Variance Acquisition

- ID: 2605
- Canonical URL: https://synthcore.org/idea/2605/decision-weighted-variance-acquisition
- API JSON: https://synthcore.org/api/idea/2605.json
- API Markdown: https://synthcore.org/api/idea/2605.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.20126](https://arxiv.org/abs/2608.20126)
- Category: training
- Solves: sample-efficiency, accuracy
- ML areas: world-model, rl, data-augmentation
- Math tags: probability, statistics, linear-algebra, optimization
- Ratings: usefulness 8/10; difficulty 5/10; novelty 6/10

## Idea description

Replace uncertainty sampling for a neural world model with acquisition scores based on the predicted reduction of downstream task loss. Query or label the state-action whose observation most reduces posterior uncertainty in the rates, rewards, or next-state quantities that affect future control decisions.

## Mathematical statement

Use a locally linear Bayesian residual model with parameters w in R^d, posterior w ~ N(mu, Sigma), feature vector phi(x) in R^d for candidate input x, and observation noise variance sigma^2. After observing x, the Gaussian covariance update is Sigma_+ = Sigma - Sigma phi(x) phi(x)^T Sigma / (phi(x)^T Sigma phi(x) + sigma^2). For task quantity r_j at rollout index j, linearize r_j approximately as r_hat_j + g_j^T(w-mu), where g_j = grad_w r_j is its sensitivity to the residual parameters and c_j >= 0 is its task importance weight. The exact reduction in weighted task variance is Delta(x) = sum_j c_j (g_j^T Sigma phi(x))^2 / (phi(x)^T Sigma phi(x) + sigma^2). Select candidates by Delta(x) divided by measurement cost. The useful property is that a rank-one posterior update gives an exact, cheap score rather than a heuristic uncertainty proxy.

## Key formulas

- $$Q_{u}^{t+1}=\left[Q_{u}^{t}-\tau\sum_{i=1}^{N}a_{iu}^{t}R_{iu}^{\star}(\bm{p}_{i}^{t})\right]_{+}$$
- $$\Sigma_{+}=\Sigma-\frac{\Sigma\phi(x)\phi(x)^{\top}\Sigma}{\phi(x)^{\top}\Sigma\phi(x)+\sigma^{2}}$$
- $$\Delta(x)=\sum_{j}c_j\left(g_j^{\top}\Sigma g_j-g_j^{\top}\Sigma_{+}g_j\right)=\sum_{j}c_j\frac{\left(g_j^{\top}\Sigma\phi(x)\right)^{2}}{\phi(x)^{\top}\Sigma\phi(x)+\sigma^{2}}$$
- $$x^{\star}=\arg\max_{x\in\mathcal{C}}\frac{\Delta(x)}{\operatorname{cost}(x)+\epsilon}$$

## Implementation notes

Integrate this at the data-collection or replay-buffer admission point of a learned world model. Use a neural predictor R_psi(s,a) for rates, rewards, or next states, and attach a Bayesian linear residual head to its penultimate feature phi(s,a). Maintain a diagonal or low-rank posterior covariance Sigma for that head; initialize it from a prior and update it with recursive Bayesian least squares after each newly measured transition. For every candidate state-action link, run the current model and obtain phi(x). Roll out the current policy for a short horizon and compute task-sensitive gradients g_j = grad_w r_j by differentiating predicted queue evolution or cumulative reward through the residual head. Compute the score Delta(x) using the rank-one formula above, divide by estimated measurement or simulator cost, and label the top K candidates subject to spatial or temporal diversity. Pseudocode: construct candidates x; compute features phi_x; obtain rollout sensitivities g_j and weights c_j; set d_x = phi_x^T Sigma phi_x + sigma^2; set s_x = sum_j c_j (g_j^T Sigma phi_x)^2 / d_x; query the top scores; update mu and Sigma; retrain the neural backbone periodically. The paper's mathematics supplies the exact covariance reduction. Empirical components are c_j, rollout gradients, noise variance, and candidate costs. Start with a small MLP dynamics model on a synthetic 2D navigation or DeepMind Control task, comparing random, ensemble-UCB, and entropy sampling at equal labeling budgets. Success is lower task loss or control regret at equal labels, especially faster reduction of failures near policy-switching boundaries rather than merely lower global prediction RMSE.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built an exact rank-one Bayesian residual covariance scorer with decision-weighted acquisition, posterior updates, toy verification, and a controlled 30-seed comparison. The mechanism manifested: direct weighted variance reduction matched the formula to 2.1e-15 relative error, importance scaling matched exactly, and noise-denominator scaling matched to 3.0e-16 relative error. Decision acquisition reduced task-variance AUC from 10.81 for uncertainty sampling to 4.90 and from 11.46 for random sampling to 4.90; final realized task MSE was comparable to uncertainty sampling and better than random.

### Mechanism check

- Verdict: Built an exact rank-one Bayesian residual covariance scorer with decision-weighted acquisition, posterior updates, toy verification, and a controlled 30-seed comparison. The mechanism manifested: direct weighted variance reduction matched the formula to 2.1e-15 relative error, importance scaling matched exactly, and noise-denominator scaling matched to 3.0e-16 relative error. Decision acquisition reduced task-variance AUC from 10.81 for uncertainty sampling to 4.90 and from 11.46 for random sampling to 4.90; final realized task MSE was comparable to uncertainty sampling and better than random.
- Confidence: 9/10
- Limitations: This is a synthetic Bayesian linear residual experiment rather than a neural world model or closed-loop RL/navigation task. It does not test neural feature learning, rollout-gradient errors, candidate costs, diversity constraints, ensemble-UCB, or GPU behavior; the uncertainty baseline is feature-posterior variance rather than a neural ensemble.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.04365
- Paired wins: 2/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/981/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/981/bench_report.json)
- [experiment.py](https://synthcore.org/code/981/experiment.py)
- [report.md](https://synthcore.org/code/981/report.md)
- [report_bench_2026-09-01T172517.md](https://synthcore.org/code/981/report_bench_2026-09-01T172517.md)
- [results.json](https://synthcore.org/code/981/results.json)
- [Download all files as ZIP](https://synthcore.org/download/981)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.