# Self-occupancy fictitious-play sampler

- ID: 2334
- Canonical URL: https://synthcore.org/idea/2334/self-occupancy-fictitious-play-sampler
- API JSON: https://synthcore.org/api/idea/2334.json
- API Markdown: https://synthcore.org/api/idea/2334.md
- Verification status: unverified
- Source: [arXiv:2608.15258](https://arxiv.org/abs/2608.15258)
- Category: sampling
- Solves: sample-efficiency, stability, accuracy
- ML areas: diffusion-sampling, data-augmentation, rl
- Math tags: stochastic-processes, optimal-transport, dynamical-systems, probability
- Ratings: usefulness 6/10; difficulty 6/10; novelty 7/10

## Idea description

Turn the paper's self-fictitious-play process into a learned sampler for latent training examples or diffusion states. A controller network generates trajectories using a best response to a slowly updated occupancy belief, and the belief is updated from the controller's own states with an exponential occupation-measure update. The slow update prevents abrupt feedback loops while the controller continually adapts toward underrepresented regions.

## Mathematical statement

The paper starts from the controlled diffusion $dX_t=\alpha_t\,dt+\sqrt{2}\,dW_t$ and the finite-horizon optimal-control objective $u^T(0,x)=\inf_\alpha\mathbb{E}[\int_0^T c(X_t,\alpha_t)\,dt+g(X_T)]$. Here $X_t\in\mathbb{R}^d$ is the sampler state, $\alpha_t$ is its control or drift, $W_t$ is standard Brownian motion, $c$ is the running cost, and $g$ is the terminal cost. The self-fictitious-play adaptation maintains a belief measure $\mu_t$ and updates it from the current trajectory occupation measure by $d\mu_t=\eta(\delta_{X_t}-\mu_t)\,dt$, or in discrete time $\mu_{k+1}=(1-\eta)\mu_k+\eta\,\delta_{X_k}$; $\eta\in(0,1]$ is the belief-update rate and $\delta_x$ is a point mass at $x$. The paper's central quantitative conclusion is that the invariant law of this slow-fast process is close to the target monotone potential-MFG equilibrium with Wasserstein error of order $O(\sqrt{\eta})$. The implementation uses this scaling as a design hypothesis: decreasing $\eta$ should reduce stationary distribution mismatch approximately as $\sqrt{\eta}$, at the cost of slower adaptation. A neural controller $f_\theta(x,\mu_k)$ is trained to minimize an empirical rollout cost augmented by a coverage potential computed from the belief.

## Key formulas

- $$u^{T}(0,x)=\inf_{\alpha}\mathbb{E}\left[\int_{0}^{T}c(X_{t},\alpha_{t})\,dt+g(X_{T})\right],\qquad X_{0}=x,\qquad dX_{t}=\alpha_{t}\,dt+\sqrt{2}\,dW_{t}.$$
- $$d\mu_t=\eta\bigl(\delta_{X_t}-\mu_t\bigr)\,dt,\qquad\text{equivalently}\qquad \mu_{k+1}=(1-\eta)\mu_k+\eta\,\delta_{X_k}.$$
- $$\mathcal{W}_2(\bar\mu_\eta,\mu^*)\le C\sqrt{\eta},$$
- $$X_{k+1}=X_k+f_\theta(X_k,\mu_k)\,\Delta t+\sqrt{2\Delta t}\,\varepsilon_k,\qquad \varepsilon_k\sim\mathcal{N}(0,I_d).$$

## Implementation notes

(1) Integration point: add this as a latent-space sampler before minibatch construction, or as a replacement for the noise trajectory in a diffusion sampler. Let $x_k$ be a latent vector, let $\mu_k$ be represented by a reservoir of $M$ latent particles, and let a small controller network $f_\theta(x_k,h_k)$ output a drift. Compute belief features as $h_k=M^{-1}\sum_{j=1}^M\psi(z_j)$, where $z_j$ are stored belief particles and $\psi$ is a fixed random Fourier feature map or a small frozen encoder. (2) Pseudocode: initialize $x_0$ and particles $z_1,\ldots,z_M$; at each step compute $h_k$, set $a_k=f_\theta(x_k,h_k)$, sample $\varepsilon_k\sim N(0,I)$, and update $x_{k+1}=x_k+a_k\Delta t+\sqrt{2\Delta t}\varepsilon_k$. Update the belief with $\mu_{k+1}=(1-\eta)\mu_k+\eta\delta_{x_{k+1}}$, implemented by replacing or exponentially reweighting reservoir entries. Train $\theta$ using an 8--32-step rollout return combining downstream validation loss, a target-energy term, and a repulsion term based on pairwise distances to belief particles. (3) Computed from the paper: the controlled diffusion, exponential occupation update, and predicted $\sqrt{\eta}$ stationary-error scaling. Estimated empirically: the constant $C$, Wasserstein error, mode coverage, and effective sample size; use Sinkhorn distance against held-out target samples. Sweep $\eta\in\{10^{-3},3\cdot10^{-3},10^{-2},3\cdot10^{-2},10^{-1}\}$. (4) First cheap experiment: use a 2D mixture of eight Gaussians with imbalanced weights. Compare ordinary Langevin sampling, learned drift without belief feedback, and the self-occupancy sampler at equal drift evaluations. Measure negative log likelihood, recovered modes, effective sample size, and Sinkhorn distance. Success requires improved minority-mode coverage and an approximately linear relationship between $\log$ stationary error and $\log\eta$ with slope near $1/2$. A second test can use CIFAR-10 autoencoder latents for data augmentation, measuring classifier accuracy at equal generated-sample budgets.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.