Model Selection vs Model Routing
People think model selection and model routing are the same job, and they are not. Model selection is a choice you make once. You test a few models on real examples of your task, then pick one. Model routing is code that picks a model again for each request, as it comes in. You often do the first to set up the second.
Model Selection (LLM)
Picking the AI model that fits a job. You test a few first.
Model selection is choosing which AI model to use for a task. Model makers offer several sizes. Big models do hard work best, but they are slower and cost more for each word. Small models are fast and cheap, and they do simple jobs well. You choose by trying a few models on real examples of your task. Then you pick the smallest one that gets them right.
Model Routing
Sending each request to the model that fits it, cheap or strong.
Model routing picks which model should answer each request before it is sent. A simple rule might send short, easy questions to a small, cheap model. Hard ones, like long code or tricky reasoning, go to a big model. Some routers use a small model of their own to make the choice. The aim is to pay for a big model only when the task needs one.
In one line
- Model Selection (LLM)
- Picking the AI model that fits a job. You test a few first.
- Model Routing
- Sending each request to the model that fits it, cheap or strong.
These live in different corners of the glossary (AI and Development), which is exactly why they get confused.