# Poisson-Calibrated Candidate-Pool Scheduler

- ID: 2136
- Canonical URL: https://synthcore.org/idea/2136/poisson-calibrated-candidate-pool-scheduler
- API JSON: https://synthcore.org/api/idea/2136.json
- API Markdown: https://synthcore.org/api/idea/2136.md
- Verification status: failed_benchmark
- Source: [arXiv:2608.11549](https://arxiv.org/abs/2608.11549)
- Category: dynamics
- Solves: sample-efficiency, speed, accuracy
- ML areas: rl, optimizer, world-model
- Math tags: probability, stochastic-processes, control-theory, dynamical-systems
- Ratings: usefulness 8/10; difficulty 5/10; novelty 7/10

## Idea description

Replace a fixed number of randomly sampled continuous actions with a state-dependent candidate pool whose size is chosen from the predicted extreme-value error of the best candidate. If the local action deficit has order \(\|u-u^\star\|^\kappa\) in an effective dimension \(d\), the best sampled action has expected Bellman error proportional to \(N^{-\kappa/d}\). This gives an explicit stopping rule for increasing the pool only when the estimated residual action error is larger than the value-function or critic uncertainty.

## Mathematical statement

Assume that at state \(x\), an optimal action \(u^\star\) is locally isolated and \(Q_V(x,u^\star)-Q_V(x,u)\approx a_x\|u-u^\star\|^\kappa\), with sampling density \(q_x(u^\star)>0\) in effective dimension \(d_x\). For \(N\) iid candidates, the rescaled near-optimal deficits converge to a Poisson process; consequently the best-candidate deficit has scale \(N^{-\kappa/d_x}\). A practical approximation uses \(\mathbb{E}[\Delta_N(x)]\approx C_xN^{-\kappa/d_x}\), where \(C_x\) is estimated from local candidate statistics. Because the Bellman operator is a \(\beta\)-contraction, a one-step action approximation error \(\Delta_N\) produces a discounted value error bounded approximately by \(\Delta_N/(1-\beta)\). The scheduler selects the smallest \(N\) satisfying \(C_xN^{-\kappa/d_x}/(1-\beta)\leq\varepsilon_x\).

## Key formulas

- $$Q_V(x,u^\star)-Q_V(x,u)\approx a_x\|u-u^\star\|^{\kappa},\qquad \Delta_N(x):=Q_V(x,u^\star)-\max_{1\leq i\leq N}Q_V(x,U_i).$$
- $$\Delta_N(x)=\Theta\!\left(N^{-\kappa/d_x}\right),\qquad \mathbb{E}[\Delta_N(x)]\approx C_xN^{-\kappa/d_x}.$$
- $$\|\mathcal{T}V-\mathcal{T}_NV\|_\infty\leq \|\Delta_N\|_\infty,\qquad \|V^\star-V_N^\star\|_\infty\lesssim \frac{\|\Delta_N\|_\infty}{1-\beta}.$$
- $$N_x=\left\lceil\left(\frac{C_x}{(1-\beta)\varepsilon_x}\right)^{d_x/\kappa}\right\rceil.$$

## Implementation notes

1. Integration point: use this in a sampled-action actor-critic or model-predictive policy layer. At each state, draw candidate actions from the actor distribution, evaluate their Q-values with the critic, and adapt the pool size before committing the action. Start with a cheap pool of N_min candidates and estimate local geometry from the top-K candidates. 2. Pseudocode: draw U[1:N], compute q_i=Q_theta(x,U_i), sort q_(1)>=q_(2)>=..., estimate a local tail exponent by fitting log(q_(1)-q_(j)) versus log(j/N), and estimate C_x from repeated minibatch states; set kappa_hat/d_hat from the fitted slope; predict e_hat=C_hat*N^(-kappa_hat/d_hat)/(1-beta); while e_hat>epsilon_x, double N, evaluate only the new candidates, and stop at N_max. Use the selected candidate for the target and optionally backpropagate through a softmax over the final pool. 3. Computed versus estimated: the contraction factor beta and candidate-pool Bellman maximization are exact; C_x, effective dimension d_x, and curvature exponent kappa are empirical estimates, preferably maintained with exponential moving averages and clipped confidence intervals. 4. First cheap experiment: use a 2D continuous-action synthetic discounted MDP with known quadratic reward and several state-dependent action dimensions. Compare fixed N in {8,32,128} against the scheduler at equal Q-evaluation budget. The predicted signature is a log-log slope of approximately -kappa/d in the measured best-action error versus N, and the adaptive pool should stop near the smallest N for which measured rollout value error is below epsilon/(1-beta). Fit the slope on held-out states; accept the mechanism only if the measured exponent is within 20 percent of the local prediction and the value error decays with the predicted power law.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: no_effect
- Verdict: Built and numerically verified the Poisson-calibrated candidate-pool mechanism on a synthetic quadratic continuous-action problem. The predicted error exponents matched closely: observed slopes were -1.865 vs -2.0 for d=1 and -0.954 vs -1.0 for d=2; doubling ratios were 0.279 vs 0.25 and 0.517 vs 0.5. Scheduler scaling matched N proportional to epsilon^(-d/kappa), with slope -0.9998 vs -1.0 and an estimated asymptotic constant within 1.4% of the exact value. The noisy adaptive scheduler did not beat fixed N=128 in the heterogeneous mini-comparison, so the mechanism is validated but an efficiency win was not demonstrated.

### Mechanism check

- Verdict: Built and numerically verified the Poisson-calibrated candidate-pool mechanism on a synthetic quadratic continuous-action problem. The predicted error exponents matched closely: observed slopes were -1.865 vs -2.0 for d=1 and -0.954 vs -1.0 for d=2; doubling ratios were 0.279 vs 0.25 and 0.517 vs 0.5. Scheduler scaling matched N proportional to epsilon^(-d/kappa), with slope -0.9998 vs -1.0 and an estimated asymptotic constant within 1.4% of the exact value. The noisy adaptive scheduler did not beat fixed N=128 in the heterogeneous mini-comparison, so the mechanism is validated but an efficiency win was not demonstrated.
- Confidence: 8/10
- Limitations: Only a static synthetic quadratic action-selection problem was tested; no learned critic, actor distribution, multi-step RL rollout, state-dependent effective-dimension estimation, confidence intervals, or GPU implementation was evaluated. The scheduler comparison used a noisy N=8 pilot and a maximum pool of 128, likely contributing to its weaker accuracy than fixed N=128.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 1
- Paired wins: 0/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_report.json](https://synthcore.org/code/784/bench_report.json)
- [poisson_action_track.py](https://synthcore.org/code/784/poisson_action_track.py)
- [poisson_scheduler_experiment.py](https://synthcore.org/code/784/poisson_scheduler_experiment.py)
- [poisson_stage2.py](https://synthcore.org/code/784/poisson_stage2.py)
- [report.md](https://synthcore.org/code/784/report.md)
- [report_bench_2026-09-01T102410.md](https://synthcore.org/code/784/report_bench_2026-09-01T102410.md)
- [results.json](https://synthcore.org/code/784/results.json)
- [Download all files as ZIP](https://synthcore.org/download/784)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.