Phenotype-Rao-Blackwellized ES

Integrate this into an ES optimizer for a stochastic policy, a noisy input layer, or a MoE router whose perturbed parameters produce an observable phenotype. Begin with the explicit Gaussian testbed, where the optimized vector is $\theta\in\mathbb{R}^d$, the latent ES perturbation is $z$, and the observed realization is $x=\theta+\sigma z+\tau\xi$. For each population member, sample $z_i\sim\mathcal{N}(0,I)$, construct $x_i=\theta+\sigma z_i+\tau\xi_i$, evaluate the network or task to obtain scalar fitness $r_i=f(x_i)$, and accumulate either the baseline or RB gradient. Pseudocode: g=0; for i in 1..N: z=randn(d); xi=randn(d); x=theta+sigma*z+tau*xi; r=f(x); if baseline: g += r*z/sigma; else: g += r*(x-theta)/(sigma*sigma+tau*tau); theta += eta*g/N. Use antithetic pairs and identical random seeds for both methods. The Gaussian posterior is analytic. In RL or MoE settings, replace it with a learned posterior mean $m_\psi(x)\approx\mathbb{E}[z\mid x]$, trained on sampled $(z,x)$ pairs by minimizing $\|m_\psi(x)-z\|^2$, then use $r_i m_\psi(x_i)/\sigma$ in the update. Estimate empirical variance from the population and monitor bias against a large-sample finite-difference gradient. First experiment: optimize a small MLP on noisy quadratic regression and CartPole with population sizes 32 and 128, comparing vanilla ES, antithetic ES, and RB-ES at equal environment evaluations. Log coordinate variance, gradient signal-to-noise ratio, and return versus evaluations. Success means at least 25% lower estimator variance and faster return improvement at equal evaluation budget; if posterior misspecification increases bias, use a validation criterion to interpolate between RB and baseline estimators.