Estimate Your LLM Bill Before You Write the Code

I write a lot of pricing articles, and they all have the same problem: they're snapshots. A provider changes a number, and a post that was accurate in March is quietly wrong by August. Meanwhile the question people actually have isn't "what does this model cost per million tokens." It's "what is my thing going to cost me."

Those are different questions, and only one of them decays.

The per-token price is an input to an estimate, not the estimate. It's like knowing the price of lumber. Useful. Tells you nothing about what the deck costs until you know how much lumber the deck takes.

So here's the arithmetic. Once you can do this, every pricing page becomes a lookup rather than a research project, and you stop being surprised by bills.

The formula

For a single call:

cost = (input_tokens  / 1_000_000) * input_rate
     + (output_tokens / 1_000_000) * output_rate

Two things about that formula catch people out.

Output is dramatically more expensive than input. The ratio is typically 4:1 or 5:1. Anthropic's Claude Opus 5 runs $5 per million input and $25 per million output; Sonnet 5 is $3 and $15; Haiku 4.5 is $1 and $5. That pattern holds broadly across providers. So a feature that reads a lot and writes a little is cheap, and a feature that writes a lot is not. "Summarize this 50-page document" is a bargain. "Generate a 50-page document" is not.

Input includes everything you send, every time. Your system prompt. Your tool definitions. The entire conversation so far. Every retrieved chunk. People estimate the cost of the user's question and forget the 4,000-token system prompt riding along on every single request.

Count tokens for real, not with a rule of thumb

"Four characters per token" is fine for a back-of-envelope. It is not fine for a budget, and it's badly wrong for code, JSON, and non-English text — all of which tokenize much less efficiently than English prose.

Do not reach for tiktoken to count tokens for a non-OpenAI model. It's OpenAI's tokenizer. Every provider has its own, and the counts differ by enough to wreck an estimate.

Most providers offer a token-counting endpoint. Use it:

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic();

const { input_tokens } = await client.messages.countTokens({
  model: 'claude-opus-5',
  system: SYSTEM_PROMPT,
  tools: TOOLS,
  messages: [{ role: 'user', content: sampleQuestion }],
});

console.log(`Input: ${input_tokens} tokens`);

Note that it accounts for the system prompt and tool definitions too, which is exactly what the rule of thumb misses.

Run this against ten or twenty representative inputs — real ones, not the tidy example you wrote for the README — and take the mean and the 90th percentile. The mean tells you the bill. The p90 tells you whether you're going to blow a context window or a rate limit.

Output tokens you have to measure empirically. Run the feature, read usage.output_tokens off the response, collect a distribution. Your max_tokens is a ceiling, not a prediction — a request with max_tokens: 4096 that returns a 200-token answer bills you for 200.

The context growth curve, which is where chat apps die

Here's the thing that turns a reasonable-looking chat feature into an unreasonable bill.

Every turn, you resend the whole conversation. So on turn n you're paying for roughly n turns of history. Total input tokens across a conversation of n turns isn't linear — it's the sum 1 + 2 + 3 + … + n, which is n(n+1)/2. Quadratic.

Concretely, with a 1,000-token system prompt and turns averaging 500 tokens in and 500 out:

Turn	Input tokens that turn	Cumulative input
1	1,500	1,500
5	5,500	17,500
10	10,500	60,000
20	20,500	220,000
50	50,500	1,300,000

A fifty-turn conversation costs about 22 times what a ten-turn conversation costs, not five times. On Opus rates that single conversation's input alone is around $6.50, and you have thousands of users.

This is the number that ruins forecasts, and it's completely invisible on the pricing page.

Three responses, in order of how much I reach for them:

1. Cache the prefix (below). This is the big one — it makes the resend nearly free.
2. Trim or summarize old turns. Keep the last N turns verbatim, replace the earlier ones with a short summary. Some providers now do this server-side; Anthropic calls it compaction.
3. Cap the conversation. Not every product needs unbounded threads.

Lever one: prompt caching

If a large chunk of your input is identical across requests — a system prompt, a document, a fixed set of few-shot examples, tool definitions — you can cache it. Subsequent requests read from the cache at a fraction of the input price.

The economics, using Anthropic's numbers as the concrete case: a cache read costs about 0.1× the normal input rate. A cache write costs 1.25× for the default five-minute TTL, or 2× for the one-hour TTL. So with the short TTL you break even on the second request (1.25 + 0.1 = 1.35 versus 2.0 uncached) and everything after that is 90% off.

const response = await client.messages.create({
  model: 'claude-opus-5',
  max_tokens: 1024,
  system: [
    {
      type: 'text',
      text: LARGE_STABLE_PROMPT,
      cache_control: { type: 'ephemeral' },
    },
  ],
  messages,
});

console.log(response.usage.cache_read_input_tokens);      // billed at ~0.1x
console.log(response.usage.cache_creation_input_tokens);  // billed at ~1.25x
console.log(response.usage.input_tokens);                 // billed at full rate

Here's the part that determines whether caching actually works for you, and it's structural rather than a flag you set:

Caching is a prefix match. The cache key is the exact bytes from the start of the prompt up to your cache breakpoint. One byte different anywhere in that prefix and everything after it is a miss. Which means the single most expensive line of code in a lot of applications looks like this:

// Every request is now unique. Nothing caches. Ever.
const system = `You are a helpful assistant. Current time: ${new Date().toISOString()}`;

Timestamps, UUIDs, per-user IDs, unsorted JSON serialization, conditionally-included prompt sections — all of them silently destroy your cache hit rate, and none of them raise an error. You just quietly pay full price forever.

The rule: stable content first, volatile content last. Frozen system prompt, deterministic tool ordering, then the varying question. Anything dynamic goes after the last cache breakpoint, not into the header.

And verify it. cache_read_input_tokens sitting at zero across repeated requests means something in your prefix is changing. Log it.

One more thing worth knowing: there's a minimum cacheable prefix, and it varies by model — 512 tokens on Opus 5, 1,024 on Sonnet 5, 4,096 on Haiku 4.5. Below the threshold nothing caches and nothing tells you. A 3,000-token prompt caches on one model and silently doesn't on another. Check the number for the model you're actually using.

Lever two: batch processing

If the work isn't latency-sensitive — overnight enrichment, bulk classification, backfilling a dataset — most providers offer a batch endpoint at roughly half price. Anthropic's Message Batches API takes up to 100,000 requests and typically finishes within the hour.

const batch = await client.messages.batches.create({
  requests: records.map((r, i) => ({
    custom_id: `record-${i}`,
    params: {
      model: 'claude-haiku-4-5',
      max_tokens: 512,
      messages: [{ role: 'user', content: buildPrompt(r) }],
    },
  })),
});

One gotcha: results come back in arbitrary order. Key off custom_id, never off array position.

Fifty percent for accepting a delay you didn't care about is the best unforced discount in this space, and it's astonishing how many batch workloads run through the synchronous endpoint out of habit.

Lever three: route to the cheapest model that passes

Not every call needs your best model. Between Opus 5 at $5/$25 and Haiku 4.5 at $1/$5, you're looking at a 5× difference. Classification, routing, extraction, simple formatting, and "does this text mention X" all tend to run fine on the small model.

The routing pattern that works:

async function route(task) {
  if (task.type === 'classify' || task.type === 'extract') {
    return call('claude-haiku-4-5', task);
  }
  return call('claude-opus-5', task);
}

Yes, that's a hardcoded if. That's the point — it's deterministic, free, and you can read it. Do not build an LLM-powered router to decide which LLM to call; you've just added a model call to save a model call.

The discipline this requires is measurement, which is the next article's problem: you need to know that the cheap model actually passes before you route to it. Route on evidence, not on vibes. But run the experiment, because the savings are enormous and the quality difference on narrow tasks is frequently zero.

Put it in a spreadsheet

Do this before you build, not after the invoice:

per-call input tokens   = system + tools + history + retrieved + question
per-call output tokens  = measured p50 from a real run
calls per user session  = measured, and remember the quadratic if it's a chat
sessions per month      = your actual traffic estimate

monthly cost = sessions
             × calls
             × [ (in / 1M × in_rate) + (out / 1M × out_rate) ]

Then apply the levers as multipliers: cached prefix at 0.1×, batch at 0.5×, cheap-model routing at whatever fraction of calls you can move.

The first time I ran this on a feature I'd already shipped, the number was about eleven times what I'd assumed, entirely because of conversation growth I hadn't modeled. Caching the system prompt and capping history brought it back under control in an afternoon. The lesson wasn't that the model was expensive. It was that I'd never done the arithmetic.

The point

Per-token prices change. They've mostly gone down, and they'll change again next quarter, and every article anyone writes about them starts rotting the day it publishes.

The arithmetic doesn't change. Count your real tokens. Model the growth curve. Cache the stable prefix and verify the hit rate. Batch what can wait. Route down when the cheap model passes.

Do that once and you can read any pricing page in thirty seconds and know exactly what it means for you — which is the only thing you actually wanted from the pricing page in the first place.