What is Model Routing?
Model Routing: Sending each AI request to the most suitable model, often a cheap fast one for easy tasks and a stronger one for hard ones, to balance cost and quality.
Model routing means your system decides, per request, which model should handle it. Simple jobs like classification, short extraction, or formatting go to small, fast, cheap models. Complex reasoning, long documents, or high-stakes decisions go to frontier models. The routing decision can be a fixed rule by task type, a lightweight classifier, or a cascade that tries the cheap model first and escalates when confidence is low.
Routing also covers fallback: if a provider is slow or down, the request moves to an alternative model. Gateways and routing layers make this easier, but each model behaves a little differently, so every route needs its own prompts and evals rather than assuming outputs are interchangeable.
Why it matters when you build or buy AI
Running everything on the best model is the simplest way to overpay. In many workflows most steps are easy and only a few need heavy reasoning. Routing them correctly can cut cost per task substantially without hurting quality, as long as you measure quality per route. Do not add routing on day one of an MVP; add it when you have real traffic and know where the money goes.
Example
An invoice pipeline uses a small model to classify documents and extract simple fields, and only sends multi-page invoices with unusual layouts to a larger model.
Related terms
More in AI & LLMs
Book 20 minutes, or send a brief
A former CTO replies. You leave with a written plan and a fixed estimate.
Prefer LinkedIn? Connect with Harish.