The Shadow Price of Intelligence: Quality Degradation in LLM Inference as a Supply Chain Problem
arXiv:2608.23986
2026
Dynamics
2 ideas extracted · analyzed Aug 29, 2026
What the math gives to ML
The paper models inference degradation as a feedback-control problem rather than a one-shot compute-saving decision. Its transferable asset is the retry-adjusted throughput threshold: a lower-quality tier can create enough additional retries that the serving queue stops draining, even when its nominal inference cost is lower. This suggests routing and throttling policies should optimize satisfied-answer capacity and use hysteresis around the instability boundary. The same model also yields a practical cost-per-satisfied-answer objective that includes retries, memory occupancy, and SLA backlog penalties.
Ideas from this paper
✗ Mechanism failed
2026
Route requests between model-quality tiers using retry-adjusted satisfied-answer throughput instead of nominal completion throughput. Add hysteresis so degradation begins only above an upper backlog threshold and ends only after the backlog is safely below a lower threshold with negative retry-adjusted drift.
Useful8/10
Difficulty4/10
Novelty7/10
Unverified
Re-invented
2026
Replace nominal per-request serving cost with expected cost per satisfied answer, including geometric retry amplification, resident-context memory, and congestion penalties. A smaller model is selected only when its lower attempt cost remains favorable after accounting for its higher probability of failure and retry-induced occupancy.
Useful7/10
Difficulty5/10
Novelty6/10