Robust Average-Reward Markov Decision Processes: Minimax-Optimal Learning via Plug-in Reductions
arXiv:2608.06545
2026
Training
1 ideas extracted · analyzed Aug 31, 2026
What the math gives to ML
The paper identifies a sharp regime boundary for robust average-reward learning: the relevant comparison is between the target accuracy ε and the perturbation scale σH₀. Its useful transferable asset is not the tabular sample bound itself, but the prescription to adapt the learning reduction to uncertainty magnitude and to the value-function bias span. In a neural actor-critic, this suggests switching between a nominal critic target and a pessimistic robust target using an online estimate of σH₀/ε, rather than applying robustness at every training step. The resulting method is directly testable through robustness-accuracy tradeoffs and equal-data learning curves.
Ideas from this paper
Unverified
2026
Train a neural average-reward actor-critic that turns robustification on only when the estimated uncertainty scale σH₀ is comparable to or larger than the desired critic accuracy ε. In the high-tolerance regime use an ordinary nominal Bellman target; in the low-tolerance regime add a total-variation pessimism penalty proportional to the learned bias span. This avoids injecting a large robustness penalty when it is statistically unnecessary while retaining protection against transition…
Useful6/10
Difficulty5/10
Novelty6/10