Poisson Tangent Limits and Critical Policy Switching for Sampled Bellman Operators
arXiv:2608.11549
2026
Dynamics
2 ideas extracted · analyzed Sep 1, 2026
What the math gives to ML
The paper offers a nonstandard asymptotic mechanism for sampled-action Bellman operators: near-optimal candidates form a marked Poisson point process after rescaling, and the leading value error is determined by a nonlinear tangential Bellman fixed point rather than by the immediate reward gap alone. Its most transferable asset is a quantitative rule for how finite candidate pools approach continuous-action optimization, including a critical switching layer when competing action branches are nearly tied. This can be used to build adaptive candidate-pool schedulers and tie-aware sampled-action policies for actor-critic, model-based RL, and differentiable optimization layers. The key falsifiable signatures are power-law value-gap decay in pool size and a smooth policy transition at a predictable perturbation scale.
Ideas from this paper
✗ Failed on benchmark
2026
When several action branches have nearly equal Q-values, select among them using their long-horizon transition consequences rather than only noisy one-step critic values. Construct a finite sampled approximation to the paper's marked tangential Bellman operator: each candidate receives a local deficit mark and a continuation-value mark, and the branch scores are iterated through a discounted fixed point. Under a perturbation of size comparable to the finite-pool extreme-value gap, the resulting…
Useful8/10
Difficulty7/10
Novelty8/10
✗ Failed on benchmark
2026
Replace a fixed number of randomly sampled continuous actions with a state-dependent candidate pool whose size is chosen from the predicted extreme-value error of the best candidate. If the local action deficit has order \(\|u-u^\star\|^\kappa\) in an effective dimension \(d\), the best sampled action has expected Bellman error proportional to \(N^{-\kappa/d}\). This gives an explicit stopping rule for increasing the pool only when the estimated residual action error is larger than the…
Useful8/10
Difficulty5/10
Novelty7/10