Robust Asynchronous Q-Learning under Reward and State Corruption via Batching

What the math gives to ML

The paper contributes a constructive robustification of asynchronous Bellman estimation: partition samples into batches, estimate each batch mean, and aggregate the batch estimates so that a minority of adversarially corrupted reward or state observations cannot dominate the update. The transferable asset is not ordinary Q-learning, but the combination of batching, bounded-target concentration, and an explicit corruption-dependent error floor. A promising neural-network adaptation is to replace per-transition bootstrapped targets or minibatch gradients with median-of-means aggregation, especially in offline RL, distributed RL, or environments with unreliable sensors. The method is cheap, preserves the usual stochastic-estimation workflow, and yields a direct comparison between clean-target variance and corruption robustness.