Which LLM for Which Work? Budgeted Model Allocation under Uncertain Evaluation

What the math gives to ML

The paper provides a decision-focused way to handle uncertain quality estimates: instead of requiring the entire model-quality table to be identified, test whether the same budget-feasible allocation remains optimal over an explicitly defined uncertainty set. This is transferable to mixture-of-experts routing, cascaded inference, and learned model selection, where expert accuracy, latency, and cost are uncertain and the routing decision is globally coupled by a compute budget. The most useful adaptation is a nominal-versus-adversarial allocation certificate, followed by targeted evaluation only on model-input pairs that can change the allocation. This can turn uncertainty estimates into a practical routing-freezing or active-profiling mechanism rather than merely adding a generic robustness penalty.