Ensemble Routing for Prompt-Adaptive Cost-Aware LLM Inference
Large language models (LLMs) increasingly support Web-based services that must balance response quality and inference cost across heterogeneous requests. Conventional routing selects one model per prompt, while fixed ensembles cannot adapt their composition and cost to individual requests.
By Saul Urso, Emanuele Carlini, Patrizio Dazzi, Matteo MordacchiniarXiv - cs.AI
Read the full story
Originally published by arXiv - cs.AI on Nov 10, 2026.
Comments
Sign in to join the discussion.