The Per-Token Price Is the Most Useless Number on the Pricing Page

Everybody compares the wrong number.

You open two pricing pages, you find the dollars-per-million-tokens line, you put them side by side, and you pick the cheaper one. I did exactly that. Then the first real bill showed up, and it was a multiple of what my napkin math had promised. Not ten percent over. A multiple.

I have since run enough production traffic across enough providers to say this plainly: the per-token sticker price is a teaser rate. It is the APR they put in the big font. I have even done the careful version of the sticker-price analysis myself, a full teardown of Grok's API pricing down to every model, every tool call, and every batch discount, and I stand by every number in it. It still would not have predicted my bill. What you actually pay is decided by five things nobody prints on the comparison chart, and if you optimize for the sticker price you will lose money to every one of them.

Here is where the real bill comes from.

1. You are buying output, and output is the expensive half

Almost every provider charges more for tokens the model writes than for tokens you send. The ratio is usually somewhere between three and five to one. That alone breaks most people's mental model, because when you imagine "cost" you imagine your giant prompt, the part you can see and control. The prompt is the cheap half.

The expensive half is the part you do not control: how much the model decides to say back. A model that pads its answers, restates the question, and wraps everything in three paragraphs of preamble is quietly running up the output meter on every single call. Two models can have identical input prices and identical output prices, and the chattier one costs you meaningfully more for the same work, because it spends more of the expensive resource to say the same thing.

Nobody benchmarks verbosity. You should. A terse model at a higher sticker price routinely beats a cheap windbag.

2. You pay for thinking you will never read

This is the one that got me, and it is the one the 2026 pricing pages are still doing a bad job of explaining.

Reasoning models generate a pile of internal tokens before they answer. Those tokens are billed. They are billed at the output rate, the expensive rate, and in a lot of cases you never even see them because they are hidden reasoning. You asked a hard question, the model thought for two thousand tokens, wrote you a two-hundred-token answer, and you got charged for twenty-two hundred tokens of output.

On an easy classification task this is invisible. On a hard task with a reasoning-heavy model, the thinking can dwarf the answer. The sticker price told you nothing about this, because the sticker price is per token and the whole surprise is how many tokens appear out of nowhere. If you switched to a "smarter" model to improve quality and your bill jumped more than the price difference explained, this is why.

3. The cache is a discount you have to earn, not one you get

Prompt caching is real and it is the single biggest lever most teams ignore. Sending the same long system prompt or the same document on every call, and paying full freight for it every time, is the most common way I see people light money on fire.

But caching is not a free coupon. Writing to the cache usually costs more than a normal input token, and reads only pay off if you actually hit the cache before it expires. Get the pattern wrong, structure your prompts so the stable part is not at the front, or spread your traffic so thin that entries expire before the next hit, and you pay the write premium without ever collecting the read discount. You made it worse.

Caching rewards workloads with a big shared prefix and steady volume. It punishes sporadic, always-different prompts. Whether your real traffic looks like the first case or the second is a fact about your system, not about the price page, and it is worth more than any per-token comparison you will ever run.

4. Tools, search, and retrieval bill on their own meter

The moment your model stops being a text-in, text-out box and starts calling tools, a second bill opens up next to the first.

Function calls, web search, code execution, image and video understanding: these are frequently priced per invocation, separate from the tokens, and the per-thousand-calls numbers add up faster than anyone expects because agentic workloads call tools in loops. An agent that takes eight tool-steps to answer one question is eight invocations plus the tokens for every intermediate round-trip, and every round-trip re-sends the growing context. The token meter and the tool meter run at the same time, and they feed each other.

If you are building agents and you only modeled token cost, your estimate is not a little low. It is a different order of magnitude.

5. You pay full price for failure

The pricing page assumes every call succeeds and every output is used. Your production system does not work that way.

Timeouts still bill for the tokens generated before they died. A response that gets cut off at the length limit bills for every token it wrote before hitting the wall, and then you pay again for the retry. A validation check that rejects a malformed answer and re-prompts pays for both the bad answer and the good one. Rate-limit backoffs, speculative parallel calls where you keep the first to finish and throw away the rest, JSON that failed to parse: all of it is billed, none of it is on the chart, and the flakier your integration the more of your bill is pure waste heat.

I have watched retry logic quietly double a workload's cost. The model was fine. The plumbing was expensive.

The number that actually matters

Here is the contrarian part, and I mean it: stop comparing price per token. It is the wrong unit.

The only number that means anything is cost per successfully completed task. Cost to get one correct, usable, validated result out the other end. That number folds in verbosity, reasoning overhead, cache behavior, tool calls, and your retry rate, which is to say it folds in all five of the things the sticker price hides. And it reorders the leaderboard completely.

A cheap model that needs two attempts and a verbose reasoning pass to land a usable answer can cost more per finished task than a pricier model that nails it in one terse shot. This is the trap in the whole cheap-model pitch. Cheap only helps you if the model does the job the first time, and "does the job the first time" is exactly the property that per-token pricing cannot show you.

So I stopped trusting the chart. The only way to know your real cost is to instrument your own workload:

• Log input, output, and reasoning tokens separately, per call. If you cannot see reasoning tokens as their own line, you are flying blind on your most volatile cost.
• Track your cache hit rate like it is a business metric, because it is one.
• Count tool invocations per completed task, not per call.
• Measure your retry and rejection rate and multiply cost by it, because that is your true effective rate.
• Divide total spend by successful outcomes. That single number is the one to compare across models. Nothing else.

Run a real slice of your actual traffic through two candidate models for a day and read those five numbers. You will learn more than a month of staring at pricing pages, and you will occasionally find that the "expensive" model is the cheap one.

The sticker price is marketing. Your bill is physics. Measure the physics.