Picking a model for each job
Send each job to the smallest model that can do it. Below that is my design for a router that upgrades itself.
An agent that plans and decides has little in common with one that bulk-edits files. Put them all on one frontier model and you overpay for grunt work and underspend on judgment. Routing by job cuts the bill by an order of magnitude and improves quality where it matters.
Tap each role to see its model tier, why, and its trap.
Route by the job
A tier is a class of model. The specific models change constantly (see below). The tag shows where the money goes: a cheap paid model where judgment lives, free or local models everywhere else.
Three traps benchmarks won't show
1. The tool-call trap. For the local grunt tier I first picked a 7B coding model that topped the charts for its size. It edited nothing. It had no tool-call template, so it printed tool calls as plain text, and the run reported success with zero files changed. Now I filter by tool-call support first and coding score second. A smaller general model with native tool calls did better.
Verify the diff, never the transcript.
2. The free-tier trap. Free frontier models are useful, but in an agent loop speed depends on per-request overhead and rate caps more than model size. A huge free model and a small one took about the same time per step. With big prompts and a low requests-per-minute cap, multi-step sessions kept stalling. The fix was a cheap paid model on the control plane, pennies a day, with the grunt tier local and free. You're paying to avoid throttling where latency compounds.
3. The stale-list trap. Free model lineups change every few weeks, so "best free model" posts are out of date when you read them. Don't hardcode a list. At setup, query the provider's models endpoint and filter by price, context window and capabilities.
A router that upgrades itself
New models appear every week, so a hand-kept table is always a month old. The design is an agent that owns the routing table. It finds new models, tests them, compares them on live work and proposes a change. A person approves it. Tap through the four stages.
The upgrade loop
The loop costs almost nothing to run and ends with a person. Testing a new model is cheap, and trusting one unchecked is not.
Why a person approves it
A model swap quietly changes the cost and behavior of every future task until someone notices. So it sits behind human approval, with evidence attached, like any other hard-to-undo action in the factory. The agent does the tedious work and you accept or reject in one tap.
The one run taught me that an eval pipeline is code review times three. The baseline, the scorer and the comparator can each fail silently. A saturated rubric will happily crown four "winners" that are all noise. A router is only as good as its eval, so a person reads the evidence before anything changes.