Эксперимент: KL-Budgeted Poisson-Race Sampling (#1205)

{ "worked": false, "confidence": 8, "verdict": "Built and numerically verified the finite-categorical Poisson race with the exact look-ahead stopping rule. The output marginal matches Q within Monte Carlo noise and mean log2 rank increases with KL, with a small candidate-count reduction versus rejection sampling (1.599 vs 1.604 close; 4.768 vs 4.813 moderate; 9.629 vs 9.663 farther). However, the observed close-case mean log2 rank (0.489) is above the stated KL-plus-L1 upper bound (0.124), and the pure-Python race is slower than rejection sampling; no neural inference or target-computation speedup was demonstrated.", "metrics": { "baseline": "Rejection sampling: mean candidate index 1.604, 4.813, 9.663 for close/moderate/farther cases; TV errors 0.0097, 0.0065, 0.0062; runtime 3.50, 9.97, 18.40 s.", "idea": "Poisson race: mean candidate index 1.599, 4.768, 9.629; mean log2 index 0.489, 1.753, 2.646; TV errors 0.0079, 0.0066, 0.0102; runtime 4.53, 11.63, 22.23 s. Identical P=Q always stops at index 1." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 poisson_race_experiment.py", "files": [ "poisson_race_experiment.py", "results.json" ], "limitations": "Only a small finite categorical toy test was run; no Transformer, language-model logits, GPU implementation, batched/vectorized implementation, FLOP-matched target-network benchmark, or comparison at matched perplexity was tested. The theorem is existential, so failure of this direct race to meet its upper bound does not disprove the paper's theorem, but it limits the claimed practical conclusion here." }