Self-occupancy fictitious-play sampler
Implementation & benchmark of arXiv:2608.15258 — Self-fictitious-play for Potential Monotone Ergodic Mean-field Games
Source paper: Self-fictitious-play for Potential Monotone Ergodic Mean-field Games arXiv:2608.15258 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Turn the paper's self-fictitious-play process into a learned sampler for latent training examples or diffusion states. A controller network generates trajectories using a best response to a slowly updated occupancy belief, and the belief is updated from the controller's own states with an exponential occupation-measure update. The slow update prevents abrupt feedback loops while the controller continually adapts toward underrepresented regions.
Formulas
Mathematical statement
The paper starts from the controlled diffusion $dX_t=\alpha_t\,dt+\sqrt{2}\,dW_t$ and the finite-horizon optimal-control objective $u^T(0,x)=\inf_\alpha\mathbb{E}[\int_0^T c(X_t,\alpha_t)\,dt+g(X_T)]$. Here $X_t\in\mathbb{R}^d$ is the sampler state, $\alpha_t$ is its control or drift, $W_t$ is standard Brownian motion, $c$ is the running cost, and $g$ is the terminal cost. The self-fictitious-play adaptation maintains a belief measure $\mu_t$ and updates it from the current trajectory occupation measure by $d\mu_t=\eta(\delta_{X_t}-\mu_t)\,dt$, or in discrete time $\mu_{k+1}=(1-\eta)\mu_k+\eta\,\delta_{X_k}$; $\eta\in(0,1]$ is the belief-update rate and $\delta_x$ is a point mass at $x$. The paper's central quantitative conclusion is that the invariant law of this slow-fast process is close to the target monotone potential-MFG equilibrium with Wasserstein error of order $O(\sqrt{\eta})$. The implementation uses this scaling as a design hypothesis: decreasing $\eta$ should reduce stationary distribution mismatch approximately as $\sqrt{\eta}$, at the cost of slower adaptation. A neural controller $f_\theta(x,\mu_k)$ is trained to minimize an empirical rollout cost augmented by a coverage potential computed from the belief.
Implementation notes
(1) Integration point: add this as a latent-space sampler before minibatch construction, or as a replacement for the noise trajectory in a diffusion sampler. Let $x_k$ be a latent vector, let $\mu_k$ be represented by a reservoir of $M$ latent particles, and let a small controller network $f_\theta(x_k,h_k)$ output a drift. Compute belief features as $h_k=M^{-1}\sum_{j=1}^M\psi(z_j)$, where $z_j$ are stored belief particles and $\psi$ is a fixed random Fourier feature map or a small frozen encoder. (2) Pseudocode: initialize $x_0$ and particles $z_1,\ldots,z_M$; at each step compute $h_k$, set $a_k=f_\theta(x_k,h_k)$, sample $\varepsilon_k\sim N(0,I)$, and update $x_{k+1}=x_k+a_k\Delta t+\sqrt{2\Delta t}\varepsilon_k$. Update the belief with $\mu_{k+1}=(1-\eta)\mu_k+\eta\delta_{x_{k+1}}$, implemented by replacing or exponentially reweighting reservoir entries. Train $\theta$ using an 8--32-step rollout return combining downstream validation loss, a target-energy term, and a repulsion term based on pairwise distances to belief particles. (3) Computed from the paper: the controlled diffusion, exponential occupation update, and predicted $\sqrt{\eta}$ stationary-error scaling. Estimated empirically: the constant $C$, Wasserstein error, mode coverage, and effective sample size; use Sinkhorn distance against held-out target samples. Sweep $\eta\in\{10^{-3},3\cdot10^{-3},10^{-2},3\cdot10^{-2},10^{-1}\}$. (4) First cheap experiment: use a 2D mixture of eight Gaussians with imbalanced weights. Compare ordinary Langevin sampling, learned drift without belief feedback, and the self-occupancy sampler at equal drift evaluations. Measure negative log likelihood, recovered modes, effective sample size, and Sinkhorn distance. Success requires improved minority-mode coverage and an approximately linear relationship between $\log$ stationary error and $\log\eta$ with slope near $1/2$. A second test can use CIFAR-10 autoencoder latents for data augmentation, measuring classifier accuracy at equal generated-sample budgets.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.