AI model routing: cost, quality and governance decisions
Most requests to an assistant are simple and are served by the most expensive model. How routing between models works, what the research says and which choices belong to the company.
In short. Model routing places a component between the person making the request and the models, which decides where to send it. The possible saving is large and depends on the type of requests, so it cannot be promised in advance. Useful routing works on a declared quality threshold and chooses the cheapest of the models that clear it, which presupposes a measurement of quality on your own cases. The most recent research urges caution about the algorithms and gives more weight to the choice of available models.
If you look at the list of requests your colleagues made to an AI assistant over the past week, you will mostly find rewordings of emails, summaries of short documents, translations, questions about an internal procedure. Next to these there are a few hard requests, the analysis of a long contract or the comparison of three technical bids, and all of them, the former like the latter, are served by the same model, usually the most capable and the most expensive available.
It happened to us too, in the first months after we put agents into ZeroFive's daily work, and we noticed only when we read consumption by type of activity. Model routing comes from this observation and consists of placing, between the person making the request and the models, a component that decides which model to send it to. Technically it is a piece of infrastructure; for whoever governs the budget it is the difference between spending that grows with use and spending that grows with value.
The economic principle
The price per token between a frontier model and a small one, open or proprietary, can differ by an order of magnitude and often more. As long as use is limited to a few people the gap goes unnoticed. When assistants become agents, which make dozens of calls in sequence to complete a task and re-read the context each time, the same gap is multiplied by the number of steps and lands in the income statement.
Research has measured how much can be recovered. The University of California, Berkeley researchers who presented RouteLLM on the LMSYS blog in July 2024 trained a number of routers on preferences expressed by users, having them choose between a frontier model and a much cheaper open one, and reported on one of the benchmarks a cost reduction of more than 85% while keeping 95% of the stronger model's quality, whereas on two others the reduction stopped at 45% and 35%. The distance between the three numbers is the most useful piece of information, because it says the saving depends on the type of requests and cannot be promised in advance.
A telco selling AI services to its customers on top of third-party models has a margin that depends almost entirely on this choice. A company that uses AI only internally has the same problem on a smaller scale, with the difference that often nobody answers for it.
The inputs of a router
A router can decide on the basis of different elements, and it is worth knowing which ones the router you are being offered uses. The simplest is a fixed rule by type of activity, so that transcription goes to the speech model and image generation to the dedicated model. A little above that are labels describing each model, compared with the content of the request. Higher still is a classifier, itself a small model, which estimates the difficulty of the question and chooses accordingly.
These techniques all look at the request and none looks at the result. They choose the model that seems suitable without knowing whether the answer produced will then be good, and they work well as long as requests resemble those foreseen by whoever wrote the rules and labels.
There is also an input that comes before any other and has nothing to do with cost: the class of the data contained in the request. A router must first exclude the destinations not permitted for that data, and only among those remaining look for the most economical. We wrote about it in protecting data when using AI, where the routing scheme by data class is covered in full.

Cost and quality in the same decision
Saying that a request goes to the most suitable model means nothing until you specify suitable for what. A model may be preferable because it costs less, because it answers sooner, because it makes fewer mistakes on that task, and the three criteria rarely point to the same candidate.
Routing that deserves to be called intelligent works on a declared quality threshold and, among the models that clear it, chooses the cheapest. For this to be possible you need to know what quality each model produces on each type of task, and that is information no vendor can hand over ready-made, because it depends on your documents and your acceptance criteria. This is why routing comes after measurement, and anyone who has not yet defined how an output is evaluated on their own real cases can get from routing only a saving whose price they do not know.
Even with the measurement available, a router knows nothing on its first day. It learns by observing: it sends a small share of requests to a cheaper model, compares the outcome with that of the usual model, and shifts traffic when the difference is negligible. It is the same scheme as systems that learn by trial and error, with an exploration share that has a cost and that someone must have authorised.
Which framework does your company actually need?
AI Rating measures maturity across the four areas of the model and shows where to start, with priorities and estimated effort.
Start your AI RatingThe limits indicated by research
It would be wrong to present routing as a settled solution. A study published in January 2026, LLMRouterBench, compared ten routing methods again on more than four hundred thousand cases and found results very close to one another, with several recent methods, some of them commercial, failing to beat a simple baseline reliably. The same study observes that adding models yields less and less, while a careful selection of those made available to the router matters more.
For decision-makers the reading is fairly linear. The advantage comes mainly from having two or three well-chosen models, one of them small and perhaps run in-house, and from sending the small one the tasks it can handle. The sophistication of the algorithm counts for less than the discipline with which the list of models is kept up to date and quality is checked for decline.
The decisions that belong to the company
The infrastructure provider supplies the mechanism. The choices that make it governable stay with the organisation, and there are few of them.
- The quality threshold for each type of task, written and approved by the process owner.
- The list of permitted destinations for each data class, external models included.
- The share of traffic that may be used to try alternative models, and on which activities.
- Who is alerted when measured quality falls below the threshold, and what happens in that case.
- The log of routing decisions, with the model actually used for each request.
The last item also matters beyond the income statement. When a system changes model from one request to the next, the question of which model produced a given answer becomes an audit question, and for systems that fall under the AI Act's record-keeping obligations the answer must be reconstructible.
Routing in the cost plan
In the budget of an AI project, model consumption is one of the few variable items, and for that reason it is often underestimated at pilot stage, when volumes are low and prices promotional. We discussed why the forecast breaks in the AI cost nobody budgeted for and in the piece on what AI projects really cost.
Routing is the tool that turns that item from endured to governed, provided it enters the project from the start. Adding it to a system already in operation, built around the behaviour of a single model, means redoing a good part of the tests. For the same reason it is worth keeping in mind when thinking about vendor lock-in: an architecture able to route is also an architecture able to substitute.
In the Value Case we ask for before authorising a use case, the cost per transaction is calculated with an explicit assumption about which model serves which share of requests, and the sensitivity of the result to that assumption is one of the numbers we take to the committee. To think through your own split you can book an assessment meeting or start from an AI Rating, which checks in the Delivery dimension whether these elements are already in place.