Actor-Critic Learning for Extended Mean Field Control with Deterministic Policies

What the math gives to ML

The paper's transferable contribution is a deterministic policy-gradient rule for environments whose transition law and reward depend on the empirical joint state-action distribution, rather than only on an individual state and action. The key asset is the chain rule on the Wasserstein space: changing policy parameters changes every particle's action and therefore changes the population measure that enters other agents' dynamics and values. This suggests a measure-aware actor-critic module for multi-agent RL or population control, trained with particle batches and an additional gradient path through the empirical joint distribution instead of treating the population statistics as constants.