Wasserstein Policy Gradient for Entropy-Regularized Linear-Quadratic Control
arXiv:2608.07433
2026
Optimization
1 ideas extracted · analyzed Sep 1, 2026
What the math gives to ML
The paper identifies a useful policy-space geometry: update each state-conditional action distribution by transporting its mass in action space, rather than by taking a Euclidean or KL step in policy parameters. In entropy-regularized LQ control, this geometry preserves the linear-Gaussian policy family, reducing the infinite-dimensional Wasserstein policy gradient to stable finite-dimensional dynamics for the feedback gain and covariance, with global well-posedness and exponential convergence. A practical neural analogue is an actor whose mean and covariance are updated from an action-space velocity field induced by the critic and entropy, optionally projected back onto a Gaussian family. The most direct test is whether this transport update improves exploration and critic-induced actor stability over SAC-style Gaussian policy gradients at equal environment samples.
Ideas from this paper
✓✓ Beats tuned baseline
2026
Replace the usual parameter-space actor update with an action-space transport update. For every visited state, move sampled actions along a critic-improving velocity field while adding the entropy velocity, then fit the transported action cloud back to the actor's Gaussian mean and covariance. This preserves the paper's key idea that policy improvement is a Wasserstein flow over conditional action laws while remaining implementable for neural actors.
Useful7/10
Difficulty5/10
Novelty7/10