# Frank-Wolfe Mixture Policies for Safe Swarm Control

- ID: 1292
- Canonical URL: https://synthcore.org/idea/1292/frank-wolfe-mixture-policies-for-safe-swarm-control
- API JSON: https://synthcore.org/api/idea/1292.json
- API Markdown: https://synthcore.org/api/idea/1292.md
- Verification status: unverified
- Source: [arXiv:2607.22678](https://arxiv.org/abs/2607.22678)
- Category: optimization
- Solves: stability, sample-efficiency, accuracy
- ML areas: rl, moe-routing, world-model, training-dynamics
- Math tags: optimization, convex-analysis, measure-theory, control-theory, optimal-transport
- Ratings: usefulness 6/10; difficulty 6/10; novelty 7/10

## Idea description

Train a population controller as a convex mixture of neural trajectory policies, using a Frank-Wolfe step to add a new policy that minimizes the current population-cost linearization. The resulting mixture operates as a structured policy ensemble and can retain feasibility when each oracle policy satisfies the same support, action, and obstacle constraints. This is a principled alternative to directly optimizing one highly nonconvex multi-agent policy.

## Mathematical statement

The agent dynamics are $\dot{x}_i(t)=f(x_i(t),u_i(t))$, where $x_i(t)\in\mathbb{R}^{n_x}$ is agent $i$'s state, $u_i(t)\in\mathbb{R}^{n_u}$ is its control, and $f$ is the dynamics model. The empirical swarm distribution is $\rho_t^N=\frac{1}{N}\sum_{i=1}^N\delta_{x_i(t)}$. Its cost is $J_N=\frac{1}{N}\sum_i\int_0^T[\ell_0(t,x_i(t),u_i(t))+\lambda(W*\rho_t^N)(x_i(t))]dt+\int\Psi(x)d\rho_T^N(x)$, where $\ell_0$ is the individual running cost, $W*\rho(x)=\int W(x-y)d\rho(y)$ is the interaction potential, $\lambda$ is its weight, and $\Psi$ is the terminal cost. A policy induces an occupation measure $\mu$ on time-state-action triples, defined operationally by $\int\varphi\,d\mu=\mathbb{E}[\int_0^T\varphi(t,x_t,u_t)dt]$, and a terminal state measure $\nu_T$. Feasible measures obey the weak Liouville identity $\int[\partial_t\phi+\nabla_x\phi^\top f]d\mu=\int\phi(T,x)d\nu_T-\int\phi(0,x)d\nu_0$ for every smooth test function $\phi$, where $\nu_0$ is the initial distribution. For a symmetric interaction energy, linearization at the current marginal $\rho_k$ produces the oracle running cost $\ell_0(t,x,u)+2\lambda(W*\rho_k)(x)$. If $s_k$ minimizes this linearized objective over the feasible occupation-measure set $\mathcal F$, Frank-Wolfe updates $\mu_{k+1}=(1-\gamma_k)\mu_k+\gamma_ks_k$, with a standard open-loop choice $\gamma_k=2/(k+2)$. For convex objectives with curvature constant $C_J$, the standard bound is $J(\mu_k)-J(\mu^*)\le 2C_J/(k+2)$. The implementation exploits the explicit convex averaging and the population-gradient term; it estimates measures and convolutions from particles rather than solving the Liouville PDE.

## Key formulas

- $$\dot{x}_{i}(t)=f(x_{i}(t),u_{i}(t)),\qquad i=1,\ldots,N,\quad t\in[0,T].$$
- $$J_N=\frac{1}{N}\sum_{i=1}^{N}\int_{0}^{T}\Big(\ell_{0}(t,x_i(t),u_i(t))+\lambda(W*\rho_t^N)(x_i(t))\Big)dt+\int\Psi(x)\,d\rho_T^N(x),\qquad \rho_t^N=\frac{1}{N}\sum_{i=1}^N\delta_{x_i(t)}.$$
- $$\int\left[\partial_t\phi(t,x)+\nabla_x\phi(t,x)^\top f(x,u)\right]d\mu(t,x,u)=\int\phi(T,x)d\nu_T(x)-\int\phi(0,x)d\nu_0(x).$$
- $$\mu_{k+1}=(1-\gamma_k)\mu_k+\gamma_k s_k,\qquad s_k\in\arg\min_{s\in\mathcal F}\left\{\int\left[\ell_0(t,x,u)+2\lambda(W*\rho_k)(x)\right]ds+\int\Psi(x)d\nu_T^s(x)\right\},\qquad \gamma_k=\frac{2}{k+2}.$$

## Implementation notes

Integrate the method at the policy and trajectory-mixture level of a multi-agent RL or neural optimal-control system, rather than replacing the ordinary minibatch optimizer directly. Maintain $K$ neural control heads $\pi_j(u\mid t,x)$ with weights $w_j\ge 0$ and $\sum_jw_j=1$. The induced occupation measure is approximated by $\mu_k=\sum_jw_j\mu_{\pi_j}$. At outer iteration $k$, roll out the current mixture for $N$ agents and store particle states $x_i(t)$. Estimate the current population field with $q_k(x_i)=\frac{1}{N}\sum_{r=1}^NW(x_i-x_r)$; use an RBF kernel or a compactly supported repulsive kernel, and compute this efficiently with batching or a neighbor search. Freeze $q_k$ and train a fresh policy head $\pi_{K+1}$ as the Frank-Wolfe oracle against the objective $\ell_0(t,x,u)+2\lambda q_k(x)$ plus terminal cost $\Psi(x_T)$. Use model-predictive shooting, differentiable dynamics, PPO, or SAC, but enforce action bounds and obstacle constraints during this oracle solve with projection, differentiable barriers, or trajectory rejection. Add the head using $w_{K+1}=\gamma_k=2/(k+2)$ and replace old weights by $w_j\leftarrow(1-\gamma_k)w_j$. During deployment, sample a head once per episode or distill the weighted ensemble into one student policy. The Liouville identity is used as the rollout-consistency principle; it need not be discretized. Estimate $\rho_k$, convolution fields, and oracle costs empirically from particles. First test 20--50 double-integrator agents in a 2D environment containing obstacles. Compare against PPO, SAC, and a single-policy direct optimizer at equal environment steps. Track total cost, collision rate, constraint violations, and outer-iteration stability. A successful result is lower cost and fewer collisions at equal samples, or faster attainment of a fixed safe-cost threshold.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.