# Phenotype-Rao-Blackwellized ES

- ID: 1694
- Canonical URL: https://synthcore.org/idea/1694/phenotype-rao-blackwellized-es
- API JSON: https://synthcore.org/api/idea/1694.json
- API Markdown: https://synthcore.org/api/idea/1694.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.02073](https://arxiv.org/abs/2608.02073)
- Category: optimization
- Solves: speedup, stability, sample-efficiency
- ML areas: optimizer, rl, moe-routing
- Math tags: probability, statistics, optimization, stochastic-processes
- Ratings: usefulness 7/10; difficulty 5/10; novelty 8/10

## Idea description

Modify an evolutionary-strategy gradient estimator so that the observed phenotype or trajectory is used to infer the conditional mean of the latent ES perturbation. Instead of multiplying fitness by the raw perturbation, multiply it by the posterior mean perturbation given the realized input; this remains unbiased and has variance no greater than the ordinary ES estimator when the conditional model is correct.

## Mathematical statement

The paper defines the objective as $F(\theta):=\mathbb{E}_{p_{\theta}(x)}[f(x)]$, where $\theta$ parameterizes an input distribution $p_{\theta}$, $x$ is the realized input, and $f(x)$ is its evaluation. For a Gaussian ES perturbation $z\sim\mathcal{N}(0,I_d)$ and perturbed parameter $\theta+\sigma z$, the ordinary score estimator is $g_{\mathrm{ES}}=f(x)z/\sigma$. If $x$ is generated conditionally from the perturbed parameter, the Rao-Blackwellized estimator is $g_{\mathrm{RB}}=f(x)\,\mathbb{E}[z\mid x]/\sigma$. The law of total expectation gives $\mathbb{E}[g_{\mathrm{RB}}]=\mathbb{E}[g_{\mathrm{ES}}]$, while the Rao-Blackwell variance identity gives $\operatorname{Var}(g_{\mathrm{ES}})=\operatorname{Var}(g_{\mathrm{RB}})+\mathbb{E}[\operatorname{Var}(g_{\mathrm{ES}}\mid x)]$, so the RB estimator cannot have larger variance. For the directly implementable Gaussian realization model $z\sim\mathcal{N}(0,I_d)$ and $x=\theta+\sigma z+\tau\xi$, with $\xi\sim\mathcal{N}(0,I_d)$ independent, the posterior mean is $\mathbb{E}[z\mid x]=\frac{\sigma}{\sigma^2+\tau^2}(x-\theta)$, yielding $g_{\mathrm{RB}}=f(x)(x-\theta)/(\sigma^2+\tau^2)$. Here $d$ is the parameter or input dimension, $\sigma$ is the ES perturbation scale, and $\tau$ is realization-noise scale.

## Key formulas

- $$F(\theta):=\mathbb{E}_{p_{\theta}(x)}\left[f(x)\right].$$
- $$g_{\mathrm{ES}}=\frac{f(x)}{\sigma}z,\qquad z\sim\mathcal{N}(0,I_d).$$
- $$g_{\mathrm{RB}}=\frac{f(x)}{\sigma}\,\mathbb{E}[z\mid x],\qquad \mathbb{E}[g_{\mathrm{RB}}]=\mathbb{E}[g_{\mathrm{ES}}].$$
- $$\operatorname{Var}(g_{\mathrm{ES}})=\operatorname{Var}(g_{\mathrm{RB}})+\mathbb{E}\left[\operatorname{Var}(g_{\mathrm{ES}}\mid x)\right],\qquad \mathbb{E}[z\mid x]=\frac{\sigma}{\sigma^2+\tau^2}(x-\theta),\qquad g_{\mathrm{RB}}=\frac{f(x)(x-\theta)}{\sigma^2+\tau^2}.$$

## Implementation notes

Integrate this into an ES optimizer for a stochastic policy, a noisy input layer, or a MoE router whose perturbed parameters produce an observable phenotype. Begin with the explicit Gaussian testbed, where the optimized vector is $\theta\in\mathbb{R}^d$, the latent ES perturbation is $z$, and the observed realization is $x=\theta+\sigma z+\tau\xi$. For each population member, sample $z_i\sim\mathcal{N}(0,I)$, construct $x_i=\theta+\sigma z_i+\tau\xi_i$, evaluate the network or task to obtain scalar fitness $r_i=f(x_i)$, and accumulate either the baseline or RB gradient. Pseudocode: `g=0; for i in 1..N: z=randn(d); xi=randn(d); x=theta+sigma*z+tau*xi; r=f(x); if baseline: g += r*z/sigma; else: g += r*(x-theta)/(sigma*sigma+tau*tau); theta += eta*g/N`. Use antithetic pairs and identical random seeds for both methods. The Gaussian posterior is analytic. In RL or MoE settings, replace it with a learned posterior mean $m_\psi(x)\approx\mathbb{E}[z\mid x]$, trained on sampled $(z,x)$ pairs by minimizing $\|m_\psi(x)-z\|^2$, then use $r_i m_\psi(x_i)/\sigma$ in the update. Estimate empirical variance from the population and monitor bias against a large-sample finite-difference gradient. First experiment: optimize a small MLP on noisy quadratic regression and CartPole with population sizes 32 and 128, comparing vanilla ES, antithetic ES, and RB-ES at equal environment evaluations. Log coordinate variance, gradient signal-to-noise ratio, and return versus evaluations. Success means at least 25% lower estimator variance and faster return improvement at equal evaluation budget; if posterior misspecification increases bias, use a validation criterion to interpolate between RB and baseline estimators.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: no_effect
- Verdict: Built Phenotype-Rao-Blackwellized ES with the analytic Gaussian posterior mean, variance-law verification, unbiasedness checks, and a paired antithetic noisy-quadratic optimizer comparison. The predicted variance ratio sigma²/(sigma²+tau²) matched observed ratios within 0.00032 across tau values; the residual variance identity also matched closely, and nonlinear-estimator mean differences stayed within 1.21 joint standard errors. RB improved distance at 20 steps from 0.2511 to 0.2249 and final distance from 0.000159 to 0.000106, so the mechanism manifested and showed a modest optimization benefit.

### Mechanism check

- Verdict: Built Phenotype-Rao-Blackwellized ES with the analytic Gaussian posterior mean, variance-law verification, unbiasedness checks, and a paired antithetic noisy-quadratic optimizer comparison. The predicted variance ratio sigma²/(sigma²+tau²) matched observed ratios within 0.00032 across tau values; the residual variance identity also matched closely, and nonlinear-estimator mean differences stayed within 1.21 joint standard errors. RB improved distance at 20 steps from 0.2511 to 0.2249 and final distance from 0.000159 to 0.000106, so the mechanism manifested and showed a modest optimization benefit.
- Confidence: 9/10
- Limitations: Only the analytic Gaussian realization model and a small noisy quadratic were tested; no learned posterior, neural network, RL environment, MoE router, wall-clock benchmark, or population/FLOP-normalized large-scale comparison was evaluated.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.14245
- Paired wins: 3/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_report.json](https://synthcore.org/code/625/bench_report.json)
- [phenotype_rb_es.py](https://synthcore.org/code/625/phenotype_rb_es.py)
- [rb_es_experiment.py](https://synthcore.org/code/625/rb_es_experiment.py)
- [report.md](https://synthcore.org/code/625/report.md)
- [report_bench_2026-08-31T200611.md](https://synthcore.org/code/625/report_bench_2026-08-31T200611.md)
- [results.json](https://synthcore.org/code/625/results.json)
- [stage2_bench.py](https://synthcore.org/code/625/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/625)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.