How LLM API Pricing Actually Works: Tokens, Caching, Batch and Blended Prices
August 9, 2026 · AI API Index
On paper, every LLM provider prices the same way: dollars per million tokens. In practice I've seen two models with near-identical sticker prices produce bills that differ by a factor of five or more, and the difference always comes down to a handful of mechanics that pricing pages mention in the fine print. Here they are.
Start with tokens
A token is the unit a model reads and writes. In English it's roughly four characters, or three-quarters of a word. Code and JSON tokenize worse. Korean or Japanese can burn two or three times the tokens of the equivalent English sentence, which is worth knowing if your product is not English-first. When you estimate costs, count tokens with the provider's tokenizer, or the estimator built into our calculator. Word counts will mislead you.
Input and output are priced separately, and output is expensive
Output tokens usually cost four to six times more than input. Claude Opus 5 charges $5 in and $25 out; GPT-5.6 Sol is $5 in, $30 out. Why does this matter? Because workloads have shapes. A summarizer reads a lot and writes a paragraph. A code generator reads a short prompt and writes three files. Two teams using the same model at the same list price can see wildly different effective rates depending on which direction their tokens flow.
To collapse the two numbers into one for comparison, the industry uses a blended price, a weighted average assuming 3 input tokens per output token. Every blended figure on this site uses that convention. It's a decent first filter. Just don't stop there if your real ratio is far from 3:1, and most non-chat workloads are.
Reasoning tokens are billed as output
Reasoning models think before they answer, and the thinking is billed at the output rate whether or not you ever see it. A hard task with a generous reasoning budget can quietly emit thousands of thinking tokens per request. When a team tells me their bill doubled after "upgrading" models, this is almost always the cause. If a task doesn't need deep reasoning, cap the budget or pick a non-reasoning tier.
Prompt caching, the lever most teams ignore
When your requests share a long common prefix (a system prompt, a tool list, a document the user keeps asking about), providers serve the repeated part from cache at a much lower rate. The typical discount is 90%: Opus 5 cached input is $0.50 against $5.00 standard, GPT-5.6 Terra $0.20 against $2.00. DeepSeek V4 Pro goes further, charging $0.0036 cached, which is under 1% of its standard input price.
Run the numbers on a chat agent with a 20K-token system-plus-tools prefix and an 80% cache hit rate and you'll find the effective input price drops by around 70%. The catch: caching matches on prefixes. Put the stable content first and the user's message last, and never let a timestamp sneak into the system prompt. I wrote more about this in the caching guide.
Batch discounts
Most providers run an asynchronous batch tier at about half price, with results coming back within a window (24 hours is common) rather than immediately. Eval runs, backfills, nightly report jobs, bulk embedding: none of these need real-time answers, and all of them belong on batch. Prices on this site are standard-tier list prices, so assume batch halves them where it's offered.
A big context window is not free context
Flagships now advertise 1M-token windows (see the full table). The window is what the model can read, and every token of it you actually use is billed as input, on every single request. Some providers even charge a higher per-token rate once a request crosses a size threshold. If you're choosing between stuffing a corpus into context and retrieving snippets per query, that decision is worth real analysis; it often dominates everything else on the bill.
Before you commit
- Measure your actual input:output ratio on production traffic. Don't assume 3:1.
- Estimate a cache hit rate and apply cached pricing. Long shared prefix plus multi-turn usually means 60%+.
- For reasoning models, log thinking-token counts on your real tasks.
- Move latency-tolerant work to batch.
- Check prices again next month. Cuts of 20 to 80% in a single month keep happening; my change log tracks them.