TL;DR
A "token" is not a fixed amount of text. Each vendor's tokenizer cuts the same file into a different number of pieces, and you pay per piece. So $/Mtok is not comparable across vendors.
Anthropic's newest tokenizer (Sonnet 5, Opus 4.8, Fable 5) produces ~30% more tokens from the same code than their previous one. The list price did not change.
On identical files, it produces 1.36-1.73x GPT's token count. TypeScript is the worst case at 1.73x.
In effective terms, Opus 4.8's $5 / $25 behaves like $7.50 / $37.50, and Sonnet 5 after its intro window like $4.50 / $22.50.
This measures input tokenization only. Output verbosity, thinking, and caching move whole-task bills further; details near the end.
We counted the same bytes under every frontier tokenizer, using each vendor's own counting endpoints, and cross-checked the counts against real paid requests. Below are the numbers and what they do to the prices on the rate cards.
Why $/Mtok is not a comparable price
A model's bill is two numbers multiplied together:
The pricing page shows the second number and treats the first as a constant. It is not a constant. It depends on the model's tokenizer, and tokenizers differ a lot between vendors. Two models can list the same "$5.00 / 1M input tokens" and produce meaningfully different bills for the same paragraph, because one of them turns that paragraph into more tokens. Since nobody publishes tokens-per-content numbers, we measured them.
How we measured
We took 16 real fixtures: English prose, an HTML page, JavaScript, Python, TypeScript and Rust files, JSON tool schemas and tool results, Chinese chat and prose, symbol-heavy text, and our own agent system prompt. Each fixture was counted, byte for byte, with every model's production tokenizer:
Anthropic models were counted with the official count_tokens endpoint, which returns the same count Anthropic bills against.
OpenAI models were counted with the documented o200k_base tokenizer via tiktoken. For the newest models we double-checked this against production: we sent real API calls to GPT-5.1, GPT-5.5, and GPT-5.6 Sol and compared the live usage numbers with the local count, using a long-minus-short delta to cancel the request framing. All three matched o200k_base exactly.
Gemini and Grok were counted with their providers' token-count endpoints.
GPT's o200k serves as the 1.00x reference throughout, mainly because it has been frozen and publicly documented for over two years, while Claude's tokenizer is the one that changed. DeepSeek and GLM are left out of the tables entirely: we only have rough characters-divided-by-four estimates for them, not real tokenizer counts, and this post is about measured numbers.
Finding 1: same list price, ~30% more tokens
Claude Opus 4.6 and Opus 4.8 have the same $5.00 / $25.00 list price. What changed between them is the tokenizer. Sonnet 4.6 and Opus 4.6 use the old one; Sonnet 5, Opus 4.8, and Fable 5 use the new one. The table counts the same bytes with both, on Anthropic's own endpoint:
Weight those rows the way a real agent request is composed, which is mostly English system prompt, tool schemas, code, and JSON, and the new tokenizer comes out around +32% per request. The Chinese row barely moved, so the inflation is concentrated in English and code.
Sonnet 5's launch price, recalculated
Sonnet 5 launched at $2.00 / $10.00, down from Sonnet 4.6's $3.00 / $15.00, which looked like a price cut. That is an intro price, and it ends August 31, 2026. While it lasts, the lower rate slightly more than covers the extra tokens, so Sonnet 5 works out a little cheaper than 4.6 for the same code. From September 1 the price returns to $3.00 / $15.00, the extra tokens remain, and the same work will cost about a third more than it did on Sonnet 4.6 at the same list price.
Checking the counter against real bills
count_tokens is a prediction, so we also sent real paid requests with max_tokens: 1 and read usage.input_tokens, which is what invoices are based on. For the same content, Opus 4.6 billed 2,541 input tokens and Opus 4.8 billed 3,191, each matching its predicted count exactly. We ran the same check on Fable 5, the most expensive model in the lineup, and it billed 3,191 as well, identical to Opus 4.8. So Fable uses the same new tokenizer and there is no extra per-token markup hidden behind its higher list price. The whole verification cost about $0.08.
Finding 2: the gap is widest on code
The cross-vendor table uses GPT's o200k as the 1.00x reference. Every cell is that model's token count for the identical file divided by GPT's, so 1.20x means 20% more tokens than GPT. Claude's new and old tokenizers are shown side by side:
The code rows sit well above the prose rows: TypeScript at 1.73x, Rust at 1.58x, JavaScript at 1.52x, Python at 1.50x, against 1.40x for English prose. Code is most of what a coding agent processes, so for that workload the 1.50-1.73x band is the relevant one.
Why is TypeScript the worst case? Because o200k is unusually efficient on it: about 4.24 characters per token, which looks like the result of training on a lot of web JavaScript and TypeScript, where camelCase identifiers and JSX patterns compress into single tokens. On Rust its efficiency drops to about 3.51 characters per token. Claude's tokenizer is roughly equally dense on both languages, so the gap is widest exactly where GPT is strongest.
Chinese behaves differently. Claude sits around 1.45-1.55x above GPT with both the old and the new tokenizer (435 vs 433 tokens against GPT's 300 on the prose fixture), so this is a long-standing property of the Claude family on CJK text, not something the new tokenizer introduced. Gemini is actually more efficient than GPT here, at 256 tokens. Which tokenizer costs you more depends on what you write.
What that does to the price
Multiply the list price by the measured divergence and you get an effective price for processing the same work. Divergence here is the blended multiplier for a typical English coding request, normalized to GPT's o200k:
A few rows are worth a second look. Opus 4.6 and 4.8 share a list price but differ by about 32% in effective price. GPT-5.5 and GPT-5.6 Sol share the tokenizer, so their identical list prices really are identical in effect. Gemini 3 Flash runs a slightly heavier tokenizer than GPT and still remains the cheapest option by a wide margin.
For an independent data point: Ploy published a production migration to GPT-5.6 Sol this week and reported 1.70M input tokens against Claude Opus 4.8's 2.60M for the same builds, about 35% fewer. That is a whole-task bill rather than a tokenizer probe, so it also folds in model verbosity, but it points the same way.
production migration to GPT-5.6 Sol
What the input ratio does not capture
Everything above measures one thing: how many input tokens identical bytes become. A full agent task adds more variables on top, and they are big ones. How many output and thinking tokens does the model spend to reach the same result? How much context does the harness load per step? How often does it call tools or spawn subagents? How does the provider price cache reads and writes?
Two consequences are worth spelling out. First, cache traffic is billed per token too, so a tokenizer that produces 32% more tokens also makes every cache write and every cache read about 32% more expensive, and on long agent sessions cache reads are most of the bill. Second, whole-task costs can diverge far more than 1.73x in either direction once verbosity and thinking are folded in. When people report that one model "uses 2-4x the tokens" of another on agent work, that can be true for their setup even though the pure input tokenization gap in our fixtures never exceeded 1.73x. The two numbers measure different layers.
By content type, the input-side range we measured for Claude's new tokenizer against GPT's o200k is: prose, HTML, and JSON at 1.36-1.42x; code at 1.50-1.73x with TypeScript on top; Chinese and symbol-heavy text at 1.44-1.53x. We put TypeScript in the title because it is both the top of the range and the thing a coding agent processes all day, not because the whole world is 1.73x.
We also measured the per-task layer directly, in a follow-up: the same nine models given one identical drawing task, one shot each, with every attempt priced from the providers' own usage numbers. The same drawing ranged from $0.004 to $0.80 depending on model and reasoning effort, and maximum effort made three models fail to finish at all. That experiment is here: The Pelican Benchmark Is Saturated. We Made 9 AI Models Draw a MacBook Pro Instead.
The Pelican Benchmark Is Saturated. We Made 9 AI Models Draw a MacBook Pro Instead
How to compare model prices
Compare on your own content. Your language and file types set the multiplier, so run a representative sample through each tokenizer before trusting a rate card.
Treat a tokenizer change as a price change. When a vendor ships a new model at the same list price, check whether the tokenizer moved. Opus 4.6 to 4.8 is a ~32% increase with no line item on any invoice.
Measure dollars per completed task, not dollars per token. That single number folds in tokenization, verbosity, thinking, and caching at once, and the provider's usage field gives you the ground truth to compute it.
$/Mtok is still useful as an opening line. It just is not sufficient, and it is not comparable across tokenizers. Vendors could fix this tomorrow by also publishing prices per byte; until someone does, the conversion work is on you.
None of this makes one model universally right. GPT-5.x is the token-lean choice on English and code, Gemini 3 Flash is remarkably cheap in effect, and Claude models earn their place on quality even when they cost more tokens to run. Just make sure the price you compare is the one you actually pay, after the tokenizer.
Sources: Anthropic pricing (anthropic.com/pricing), OpenAI pricing (platform.openai.com/docs/pricing), Google Gemini API pricing (ai.google.dev), xAI pricing (docs.x.ai). Token counts come from Anthropic's count_tokens endpoint, OpenAI's o200k_base (verified against live API usage), and the Google and xAI count endpoints. No text was generated to produce these counts.
platform.openai.com/docs/pricing
The measurements are ours; the prose was drafted with AI assistance and edited by a human. Updated 2026-07-14 with a TL;DR, tighter wording, and a section on what input tokenization does not capture, based on reader feedback.
Playcode keeps every one of these models one click apart, so you can run the same prompt on two of them and compare the result that matters, the app it builds, instead of arguing about a sticker. Try it at playcode.io.