Cost per satisfied answer routing

Integrate this objective into a two-tier or multi-tier LLM router that chooses among models with different latency, correctness, and retry rates. For every tier j, measure mean attempt latency E[S_j], GPU energy or dollar cost per attempt c_attempt_j, and retry probability q_j. Estimate q_j with exponentially weighted counts of retries attributable to each tier, and clip it to at most 1 minus a small numerical constant. Compute M_j = 1 divided by (1 - q_j) and effective service time tilde S_j = M_j E[S_j]. For the current backlog N, calculate the marginal memory cost gamma c_m ((N + 1)^kappa - N^kappa), and calculate the finite-difference increase in SLA penalty using the paper's cost expression. Add expected retry-amplified compute cost, memory cost, and SLA cost, then divide by the estimated probability of obtaining a satisfactory answer. Enforce a minimum quality constraint, such as predicted correctness at least Q_min, and use small exploration probability so a tier does not become statistically unmeasurable. The mathematical quantities are the geometric multiplier and congestion cost; retry attribution, quality prediction, and cost estimates are empirical. Start with a 7B/1.5B pair on GSM8K, MMLU, or a replayed production-like prompt trace under fixed GPU capacity. Compare token-cost routing and latency-only routing. Measure dollars per correct answer, GPU-hours per satisfied answer, retry volume, KV-cache occupancy, queue length, and accuracy. The falsifiable hypothesis is that the small model's apparent cost advantage disappears during congestion once retries and resident-context costs are included.