Stop Pricing Your Agent at Its Hardest Step

The invoice is the surprise, not the demo. Across the agent teams we work with, the shape repeats often enough to be predictable: a system has shipped, it is working, and it is quietly costing several times what anyone modeled. Open up where the money actually goes and the breakdown is almost comic. A run of roughly twenty steps, and most of them are clerical. Parse the incoming ticket. Pull three fields out of it. Decide which of four tools to call. Format the reply into the template. Not one of those steps needs a model that can reason about a novel problem, and every single one of them is being answered by the largest, most expensive model in the catalog, because that is the model the agent was built on and nobody ever gave any step a reason to use a different one.

That is not a pricing problem you can wait out, and it is not a model problem. It is an architecture problem with a name, and once you can see it you cannot unsee it in your own systems.

The peak-demand tax

Every step in an agent has a demand: how much capability it actually requires to be done correctly. Extracting a date from a sentence has a low demand. Deciding, mid-run, that the customer’s request is contradictory and the plan has to change has a high one. The trouble is that almost nobody sizes an agent to the distribution of its steps. They size it to the maximum. One frontier model is chosen, it is provisioned to clear the single hardest thing the agent will ever be asked to do, and then every other step, however trivial, is billed at that ceiling rate.

Call it the peak-demand tax: you pay top-tier prices on the clerical majority of your steps in order to cover the rare step that genuinely needs the ceiling. It is the same mistake as renting a moving truck for your daily commute because once a year you move a sofa. The truck is not the wrong tool for the sofa. It is the wrong tool for the other three hundred and sixty-four days, and the cost of that mismatch does not show up in the demo, where a single flawless run looks like proof the setup is correct. It shows up on the invoice, at scale, once the thing runs a hundred thousand times.

An agent priced at its hardest step pays frontier rates for clerical work. The demo runs once and looks flawless; the tax is charged a hundred thousand times.

The reason this is worth naming is that the reflexive fixes all aim at the wrong thing. Waiting for frontier models to get cheaper lowers the unit price of the over-provisioning without removing the over-provisioning; you are still paying the ceiling rate for a date extraction, just a slightly smaller ceiling rate. Negotiating a better contract does the same. Neither touches the actual defect, which is that the work and the tool were never matched in the first place.

You cannot route what you have not decomposed

Here is the part that catches teams off guard. When they finally decide to put a cheaper model on the easy steps, they discover they cannot, because there are no “steps” in their system to point a router at. The agent is a single loop with one model and one prompt, and the eighteen clerical operations and the two hard ones are all dissolved into the same undifferentiated stream of calls. There is nothing to right-size because nothing has been separated out to size.

This is the undecomposed loop, and it is the real blocker far more often than model capability is. The symptom is that you can describe what your agent does at the level of the whole task (“it resolves refund tickets”) but not at the level of the individual step, with each step’s inputs, its output shape, and how you would know if it went wrong. If you cannot write that description, you cannot route, because routing is nothing more than deciding, per step, which model is cheapest among those that can meet that step’s described demand. The description is the prerequisite. Right-sizing is not a model-shopping exercise you do at the vendor’s pricing page; it is a decomposition discipline you do in your own code, and the shopping only becomes possible once the decomposition is done.

The industry’s framing works against this, because the loud debate is “SLM versus LLM,” as if the decision were a single global choice of camp. It is not. The NVIDIA research group made the sharpest version of the counter-argument last year, and the useful part is not the slogan on the cover; it is the specific claim underneath it. A small model, one they put at roughly seven billion parameters, is not a worse frontier model. It is a genuinely sufficient tool for the narrow, repetitive, structured operations that make up the bulk of what agents actually do, and it is dramatically cheaper to serve.

The conclusion the same authors reach is the one that matters for architecture: where general-purpose ability is genuinely needed, the right shape is a heterogeneous system, an agent that invokes more than one model, calling the small one for the many easy steps and the large one for the few hard ones. The question was never which model your agent should be. It is which model each step should get.

What routing actually looks like

Decompose the loop and the mechanism becomes concrete. An orchestrator sits in front of the models. For each step it decides, from the step’s known demand, which tier to send the work to, checks a cache so identical sub-tasks are never paid for twice, and only escalates to the frontier model when the step’s demand genuinely requires it. The model still does the thinking inside each step. The orchestrator decides, deterministically, which model is even asked.

sequenceDiagram
    participant O as Orchestrator
    participant C as Cache
    participant S as Small model
    participant F as Frontier model
    O->>C: Seen this step before?
    C-->>O: Miss
    Note over O: Classify step by demand
    alt Routine step (extract, pick tool, format)
        O->>S: Run step
        S-->>O: Result
    else Hard step (replan, resolve ambiguity)
        O->>F: Run step
        F-->>O: Result
    end
    O->>C: Store result

The classification box in the middle is where the work is, and it is worth being precise about, because “route the easy steps to the cheap model” is useless advice until you can say what makes a step easy. That is what the demand profile is for. Score each step on four axes. Reasoning depth: does the step chain several inferences, or is it a single transformation? World-knowledge breadth: does it need broad general knowledge, or only the narrow context already in front of it? Output-structure rigidity: is the output a strict, checkable shape like JSON or a tool call, or open prose? Error tolerance: is a mistake cheap to catch and retry, or does it silently corrupt the run? A step that is shallow, narrow, structured, and tolerant is exactly what a small model is good at, and there are usually a lot of them. A step that is deep, broad, open, and unforgiving is where the frontier model earns its rate, and there are usually few. The profile turns a vague instinct into a decision you can defend in review and, more importantly, one you can measure.

When frontier-by-default is the right call

None of this is a rule that smaller is always better, and treating it as one is its own failure mode. There are honest cases where routing every step to the frontier model is correct, and pretending otherwise wastes engineering effort chasing a saving that is not there.

That last condition is the one most likely to be skipped, and it is where the second anti-pattern lives. Premature routing is moving steps onto a cheaper model before you have a way to measure whether quality survived the move. The symptom is a cost graph that drops satisfyingly while a quality signal you were not watching quietly degrades, and by the time a customer notices, the run has been producing subtly worse results for weeks. Right-sizing without per-step evaluation is not optimization; it is gambling with a delayed tell. The eval comes first, then the down-tiering, never the other way around.

The constraints nobody puts on the slide

Two frictions decide whether this holds up in practice. The first is that a router and a set of per-step evaluations are real engineering, not a config flag, and they have to be maintained as the agent changes. Every new step needs a demand profile; every model swap needs its evals re-run. This is why volume is the gating variable. Below some throughput the tax you are avoiding is smaller than the machinery you are building to avoid it, and the honest answer is to leave it alone until the volume justifies it.

The second is the one that most resembles the story we opened with. Decomposing an existing monolithic agent into steps with named demands is work that produces no new feature, and it competes for time against things that do. It is tempting to defer it, to keep shipping on the single frontier model and treat the bill as a cost of doing business, right up until the bill becomes the business’s problem and the decomposition happens under pressure instead of on purpose. The teams that avoid that are the ones who treated the step, not the whole agent, as the unit of design from the start, so that right-sizing was a matter of pointing a router at boundaries that already existed rather than carving boundaries into a system that never had any.

The agent demo that dazzles the room is a claim about capability, and capability, sized to the hardest step, is exactly what makes the invoice arrive looking the way those invoices do. Production is a claim about fit: the right model on the right step, most of them cheap, a few of them not, each chosen because the work in front of it was described well enough to choose. The teams whose agent economics work are not the ones who found a cheaper frontier model or waited for one. They are the ones who stopped pricing the whole agent at its hardest step, decomposed the loop into steps they could actually name, and paid, per step, for exactly the capability the step required.

If your own agent bills are climbing and the reflex is to wait for cheaper frontier models, the more useful move is usually to look at what each step actually demands. We help teams decompose an agent into steps with known requirements, put the small model where the work is routine, and reserve the frontier model for the steps that genuinely need it, without down-tiering blind. If that is the decision in front of you, we are glad to compare notes.