{
 "artifacts": null,
 "category": "optimization",
 "description": "Replace nominal per-request serving cost with expected cost per satisfied answer, including geometric retry amplification, resident-context memory, and congestion penalties. A smaller model is selected only when its lower attempt cost remains favorable after accounting for its higher probability of failure and retry-induced occupancy.",
 "formulas_latex": [
  "$$\\tilde{S}_{j}=M_{j}\\,\\mathbf{E}[S_{j}]$$",
  "$$q_j=d_j\\rho,\\qquad M_j=\\frac{1}{1-q_j}=\\frac{1}{1-d_j\\rho},\\qquad \\tilde S_j=\\frac{\\mathbf{E}[S_j]}{1-d_j\\rho}$$",
  "$$c(k,N)=\\gamma\\bigl[k w_{0}+N\\,\\Delta w_{j}\\bigr]+\\gamma c_{m}N^{\\kappa}+C_{\\mathrm{SLA}}\\left(\\frac{(N-mk)^{+}}{mk}\\right)^{\\beta}$$",
  "$$\\mathrm{score}_j=\\frac{M_j c^{\\mathrm{attempt}}_j+\\Delta c^{\\mathrm{memory}}_j+\\Delta c^{\\mathrm{SLA}}_j}{1-q_j}$$"
 ],
 "id": 121,
 "implementation": "Integrate this objective into a two-tier or multi-tier LLM router that chooses among models with different latency, correctness, and retry rates. For every tier j, measure mean attempt latency E[S_j], GPU energy or dollar cost per attempt c_attempt_j, and retry probability q_j. Estimate q_j with exponentially weighted counts of retries attributable to each tier, and clip it to at most 1 minus a small numerical constant. Compute M_j = 1 divided by (1 - q_j) and effective service time tilde S_j = M_j E[S_j]. For the current backlog N, calculate the marginal memory cost gamma c_m ((N + 1)^kappa - N^kappa), and calculate the finite-difference increase in SLA penalty using the paper's cost expression. Add expected retry-amplified compute cost, memory cost, and SLA cost, then divide by the estimated probability of obtaining a satisfactory answer. Enforce a minimum quality constraint, such as predicted correctness at least Q_min, and use small exploration probability so a tier does not become statistically unmeasurable. The mathematical quantities are the geometric multiplier and congestion cost; retry attribution, quality prediction, and cost estimates are empirical. Start with a 7B/1.5B pair on GSM8K, MMLU, or a replayed production-like prompt trace under fixed GPU capacity. Compare token-cost routing and latency-only routing. Measure dollars per correct answer, GPU-hours per satisfied answer, retry volume, KV-cache occupancy, queue length, and accuracy. The falsifiable hypothesis is that the small model's apparent cost advantage disappears during congestion once retries and resident-context costs are included.",
 "math_summary": "For tier j, S_j is the service time of one attempt, M_j is the expected retry multiplier, and E[S_j] is the mean attempt service time. The effective service time is tilde S_j = M_j E[S_j]. If q_j = d_j rho is the probability that an attempt causes another retry and q_j is less than one, geometric retrying gives M_j = 1 divided by (1 - q_j). The paper's instantaneous cost is c(k,N) = gamma [k w_0 + N Delta w_j] + gamma c_m N^kappa + C_SLA (((N - mk)^+) / (mk))^beta. Here gamma is electricity price, w_0 is idle server power, Delta w_j is active slot power, c_m N^kappa is resident-memory overhead, C_SLA is the congestion penalty coefficient, and beta is its exponent. Use the resulting marginal cost per satisfied answer as a routing score.",
 "math_tags": [
  "probability",
  "optimization",
  "dynamical-systems"
 ],
 "ml_areas": [
  "moe-routing",
  "scheduler",
  "inference-speedup",
  "kv-cache"
 ],
 "paper": {
  "arxiv_id": "2608.23986",
  "arxiv_url": "https://arxiv.org/abs/2608.23986",
  "summary_what_math_gives_to_ml": "The paper models inference degradation as a feedback-control problem rather than a one-shot compute-saving decision. Its transferable asset is the retry-adjusted throughput threshold: a lower-quality tier can create enough additional retries that the serving queue stops draining, even when its nominal inference cost is lower. This suggests routing and throttling policies should optimize satisfied-answer capacity and use hysteresis around the instability boundary. The same model also yields a practical cost-per-satisfied-answer objective that includes retries, memory occupancy, and SLA backlog penalties.",
  "title": "The Shadow Price of Intelligence: Quality Degradation in LLM Inference as a Supply Chain Problem",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 6,
  "usefulness": 7
 },
 "solves": [
  "accuracy",
  "scalability",
  "memory"
 ],
 "title": "Cost per satisfied answer routing",
 "url": "https://synthcore.org/idea/121/cost-per-satisfied-answer-routing",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}