Q-Learning with Stable Infinite-Dimensional Linear Function Approximation

What the math gives to ML

The paper isolates a constructive way to avoid the deadly triad: place the learning variable in a latent function space and require reconstruction and compression to be nonexpansive in the sup norm. Their composition turns the Bellman update into a contractive latent map, while smoothness over a compact latent metric space controls uniform estimation error through covering numbers rather than ambient parameter dimension. This suggests replacing an unconstrained neural Q-head with a convex, partition-of-unity decoder and a normalized positive compression operator. The most direct experiment is a stable Q-learning head on top of frozen neural features, comparing divergence frequency and sample efficiency against an unconstrained linear or MLP head.