Your bill is two numbers multiplied by two rates. Both numbers are measurable before you ship.
Highlights
- A token is the unit you are billed in, and it is neither a word nor a character, which is why estimates built on either one drift.
- The common rule of four characters per token was wrong by anywhere from minus 67 to plus 46 percent across the samples measured here.
- Output tokens cost two to six times what input tokens cost across every major provider, so a chatty system has a different cost shape from a summarising one.
- Counting only your message content underestimates the bill, because every request carries roughly seven to nine tokens of envelope on top.
- Resending conversation history is the cost surprise that catches most teams, and ten turns cost 5.1 times the text that actually exists.
- Prompt caching is priced at about a tenth of normal input, which is the largest discount available and the one most systems never claim.
- The
usagefield on every response gives you exact billed counts, so you can stop estimating the moment you have made one real call.
A model API bill is simpler than it looks. It is your input tokens times an input rate, plus your output tokens times an output rate.
Both counts can be measured before you ship, and almost nobody does it. So a system works fine in testing and then produces a bill nobody predicted. Usually from text that was being sent over and over, unnoticed.
This post measures the parts people get wrong. Every number below came from a real run rather than an estimate.
What a token actually is
A token is a chunk of text the model treats as one unit. Usually part of a word. It is also the unit you are billed in.
Common words are one token. Rarer words split into several. Punctuation, spaces and line breaks all count. So your token count depends on what the text is made of, not just how much there is.
That is the part people skip, and it is where estimates go wrong.
The rule of thumb, tested
You will see "about four characters per token" everywhere. Here is that rule against real samples, using the tokenizer OpenAI models use.
Here is the script that produced it.
import tiktoken
encoding = tiktoken.get_encoding("o200k_base")
SAMPLES = {
"plain English": "The service will not start and the container keeps restarting after every deploy.",
"with a URL": "See https://kodekloud.com/blog/how-to-use-function-calling-with-python/ for details.",
"JSON payload": '{"model": "claude-haiku-4-5", "messages": [{"role": "user", "content": "hi"}]}',
"Python code": "def cosine(a, b):\n return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))",
"kubectl command": "kubectl get pods --all-namespaces -o jsonpath='{.items[*].metadata.name}'",
"a UUID": "b1cdc5e5-9f06-4cbb-83b6-6792e511f8a6",
"German": "Der Dienst startet nicht und der Container startet nach jedem Deployment neu.",
}
for name, text in SAMPLES.items():
tokens = len(encoding.encode(text))
print(f"{name:18s} {len(text):5d} chars {tokens:4d} tokens "
f"{len(text) / tokens:6.2f} chars/token")Read the spread, not the average. The rule overestimates plain prose by more than 40 percent. It underestimates a UUID by 67 percent.
That UUID row is worth sitting with. Thirty six characters, one "word", and 27 tokens. Identifiers, hashes and timestamps tokenize badly. The model has never seen them, so it has to spell them out.
So if your prompts carry IDs, log lines, JSON or code, the rule tells you costs are lower than they are. If you send plain prose, it tells you they are higher.
Counting properly
Counting takes one package and two lines.
pip install tiktokenimport tiktoken
encoding = tiktoken.get_encoding("o200k_base")
text = "The service will not start and the container keeps restarting."
print(len(encoding.encode(text)))o200k_base is the encoding current OpenAI models use. Older ones used cl100k_base. Other providers have their own, so a count from one tokenizer is an approximation for another.
That approximation is fine for budgeting. You are telling 200,000 tokens a day from 2,000,000, not billing anyone.
Watch out
Counting your message content undercounts the bill, because every request carries an envelope you are also charged for. Measured against a real API, a two token message billed as nine, an eleven token message billed as twenty, and a twenty token pair billed as twenty nine. So roughly seven to nine tokens per request go to role markers and message structure. That is noise on a long prompt and most of the bill on a short one, which matters if you are sending many tiny requests rather than a few large ones.
The asymmetry that shapes your bill
Every provider charges more for output than input, and the gap is large.
Rates as published in mid 2026. Check the provider's own page before budgeting. These move often, and published comparisons disagree with each other more than you would expect.
The multiple is what to plan around. A system that reads long documents and returns short answers is cheap. One that takes short prompts and writes long replies is expensive, even though it feels lighter.
So the question that sets your bill is not how much you send. It is how much you ask for back.
The cost that surprises people
Here is the one that catches teams after launch.
A chat API has no memory. To continue a conversation you resend the whole history each turn. So turn ten does not cost one turn. It costs everything said so far.
system = count("You are a support assistant for a Kubernetes platform team. Answer "
"using only the provided runbook context. Cite the page number for "
"each fact. If the context does not contain the answer, say so.")
question = count("My pod keeps crashing, what should I check first?")
answer = count("Check the previous container logs with kubectl logs --previous, "
"because the current container may have only just started.")
running = 0
for turn in range(1, 11):
sent = system + turn * (question + answer) - answer
running += sent
print(f"{turn:5d} {sent:8d} {running:8d} {system * turn:8d}")Ten turns of a conversation containing 564 tokens of actual text cost 2,890 input tokens. That is 5.1 times the text that exists.
The system prompt is the quiet part. Forty four tokens, sent every turn, so 440 tokens by turn ten. A generous system prompt written once becomes a charge on every message, forever.
That growth is quadratic. Twenty turns is not twice ten turns, it is roughly four times.
Three things follow from that shape, and all of them are cheap to do.
Trim the history. Most conversations do not need turn one by turn fifteen. The last few turns plus a short summary is usually enough.
Keep the system prompt tight. Every word in it is paid for on every request. No other text in your application is paid for as often.
Turn on prompt caching. Cached input is priced at roughly a tenth of normal input across providers, and a system prompt that never changes is exactly what caching is for.
Quick tip
Prompt caching is the largest discount available and the one most systems never claim. Cache reads cost about ten percent of normal input, so a stable system prompt sent a thousand times costs a tenth as much. It works because the provider keeps the processed prefix rather than reprocessing identical text. The catch is that it matches on prefixes, so anything varying must go after the stable part. Putting a timestamp at the top of your system prompt quietly disables it.
Practise the whole pipeline
Cost is one of the things that decides whether an AI feature ships. The AI Agents Fundamentals course covers tokens, embeddings, retrieval, orchestration and MCP through an end to end project, so this sits in context rather than on its own.
Stop estimating once you can measure
Every response carries a usage block with exactly what you were billed.
response = client.chat.completions.create(model=MODEL, messages=messages)
usage = response.usage
print(usage.prompt_tokens, usage.completion_tokens, usage.total_tokens)Two fields in there matter more than the totals.
cached_tokens tells you whether caching works. If you enabled it and this stays at zero, something before the cache point is changing between requests. You are paying full price for text you thought was free.
reasoning_tokens appears on reasoning models and is billed as output. You never see it in the reply. So a model that thinks at length can cost several times what the visible answer suggests.
Once you have made one real call, stop estimating. Log the usage block and your cost model becomes measurement.
A cost estimator you can run
Put it together and you get something worth keeping.
import tiktoken
encoding = tiktoken.get_encoding("o200k_base")
RATES = {
"claude-haiku-4-5": {"input": 1.00, "output": 5.00, "cached": 0.10},
"claude-sonnet-5": {"input": 3.00, "output": 15.00, "cached": 0.30},
"gpt-5.6-sol": {"input": 5.00, "output": 30.00, "cached": 0.50},
"gemini-3.1-pro": {"input": 2.00, "output": 12.00, "cached": 0.20},
}
ENVELOPE = 9
def count(text):
return len(encoding.encode(text))
def estimate(system, avg_question, avg_answer, turns, requests_per_day, model):
rate = RATES[model]
system_tokens = count(system)
question, answer = count(avg_question), count(avg_answer)
sent = 0
for turn in range(1, turns + 1):
sent += system_tokens + turn * (question + answer) - answer + ENVELOPE
produced = answer * turns
daily_in = sent * requests_per_day
daily_out = produced * requests_per_day
cost_in = daily_in / 1_000_000 * rate["input"]
cost_out = daily_out / 1_000_000 * rate["output"]
print(f"{model}")
print(f" input {daily_in:>10,} tokens/day ${cost_in:>7.2f}")
print(f" output {daily_out:>10,} tokens/day ${cost_out:>7.2f}")
print(f" total ${cost_in + cost_out:>7.2f}/day")
print(f" monthly ${(cost_in + cost_out) * 30:>7.2f}")
SYSTEM = ("You are a support assistant for a Kubernetes platform team. Answer using "
"only the provided runbook context. Cite the page number for each fact.")
for model in RATES:
estimate(SYSTEM, "My pod keeps crashing, what should I check?",
"Check the previous container logs with kubectl logs --previous.",
turns=10, requests_per_day=500, model=model)Notice the spread. The same workload costs several times more on one model than another. And the ranking depends on your output volume, not on which model is best.
That is the useful result. Not a precise bill, but knowing which order of magnitude you are in, and which model choices change it.
In the wild
The estimate that matters is the one you make before writing the feature, not after. A system that returns long prose answers and one that returns a structured summary can differ by an order of magnitude, and that is a design decision rather than a tuning problem. Deciding it after launch means rewriting. Twenty minutes with a tokenizer, before anything is built, tells you whether your idea is affordable at the volume you expect, and occasionally tells you to build something different.
What to check before you ship
Five things, in about an hour.
Count your real prompts rather than a sample sentence, using text with your actual identifiers, JSON and code in it.
Estimate your output volume honestly. It is the expensive side. People underestimate it because the reply feels shorter than the prompt.
Work out what your history strategy costs at the conversation length you expect, not at one turn.
Check whether caching applies, then confirm it is working by watching cached_tokens on real traffic.
Set an alert on daily spend at about twice your estimate. If the estimate is right you never hear from it. If it is wrong you find out in a day, not at the end of the month.
What you should be able to answer now
Why is the four characters per token rule unreliable? Because token count depends on what your text is made of. It overestimated plain prose by 43 percent here and underestimated a UUID by 67 percent.
Which side of the bill should you plan around? Output, since it costs two to six times input everywhere. What you ask for back matters more than what you send.
Why does a ten turn conversation cost so much more than ten single questions? Because you resend the entire history every turn, so the cost grows quadratically. Ten turns cost 5.1 times the text that exists.
When should you stop estimating? After your first real call. The usage block reports exactly what you were billed, so logging it turns a guess into a measurement.
The whole bill is two numbers times two rates. Both are measurable today, on a laptop, before anything is built. Those twenty minutes are often the difference between a feature that ships and one switched off in month two.
Ready to Build the Rest of the Pipeline?
Cost sits alongside retrieval quality and reliability in deciding whether something ships. The AI Agents Fundamentals course covers tokens, embeddings, retrieval and orchestration end to end. The AI Learning Path sequences that alongside vector databases, MCP and agents. Start by counting one real prompt.
FAQs
Q1: How many tokens are in a word?
There is no fixed ratio. That is the whole problem with rules of thumb. Measured on real samples, plain English ran about 1.08 tokens per word. A single UUID was 27 tokens for what looks like one word. A JSON payload came out at 2.64 characters per token against 5.79 for plain prose. Common words are usually one token. Rare words, identifiers, hashes and code split into many. The tokenizer has to spell out anything it has not seen as a unit. So the ratio depends on what your text is made of. If your prompts carry IDs, log lines or code, any rule based on words or characters will underestimate your bill. Count your real prompts with a tokenizer instead. Two lines, and the guesswork is gone.
Q2: How do I count tokens in Python?
Install tiktoken and encode your text. Two lines: encoding = tiktoken.get_encoding("o200k_base") then len(encoding.encode(text)). That encoding is what current OpenAI models use. Older ones used cl100k_base. Other providers have their own, so a count from one is an approximation for another. It is close enough for budgeting, since you are telling 200,000 tokens a day from two million rather than billing anyone. One thing to know. Counting only your message content undercounts what you are charged. Measured against a real API, a two token message was billed as nine, and a twenty token pair as twenty nine. So roughly seven to nine tokens per request go to role markers and message structure. Add that envelope to your estimates, especially if you send many small requests.
Q3: Why do output tokens cost more than input tokens?
Because generating text costs more than reading it. Input can be processed in parallel across the whole prompt. Output has to be produced one token at a time, each depending on the last. Providers price that difference, and it is consistent. Claude Haiku is five times, gpt-5.6-sol is six times, gemini-3.1-pro is six times, and DeepSeek is two times. This changes how you design, not just how you budget. A system reading long documents and returning short answers is cheap. One taking short prompts and writing long replies is expensive, even though it feels lighter to use. So when you want to cut cost, look at what you ask the model to produce before you look at what you send it.
Q4: Why did my costs grow faster than my usage?
Almost certainly conversation history. A chat API has no memory, so continuing a conversation means resending everything said so far, every turn. Turn ten does not cost one turn. It costs the whole conversation up to that point. Measured here, ten turns of a conversation containing 564 tokens of real text cost 2,890 input tokens, which is 5.1 times the text that exists. The growth is quadratic, so twenty turns is roughly four times ten turns rather than twice. The system prompt is the quiet part. A 44 token prompt sent every turn is 440 tokens by turn ten. Three fixes help. Trim old turns and keep a summary. Tighten the system prompt. And enable prompt caching, which prices repeated prefixes at about a tenth of normal input.
Q5: What is prompt caching and why does it matter?
It is the largest discount most APIs offer, and the one most systems never claim. Cached input costs about a tenth of normal input. So a stable system prompt sent a thousand times costs a tenth as much. It works because the provider keeps the processed form of a repeated prefix, rather than reprocessing identical text every request. The catch is that it matches on prefixes. Everything before the varying part must be byte for byte identical. Put a timestamp or a request id at the top of your system prompt and it quietly stops working. Nothing tells you. So check the cached_tokens field on real traffic. If you enabled caching and it stays at zero, something before the cache point is changing, and you are paying full price for text you thought was free.
Q6: How accurate do my cost estimates need to be?
Accurate enough to tell one order of magnitude from another. That is a lower bar than it sounds. You want to know whether a feature costs ten dollars a month or ten thousand, and whether a model choice changes that. A tokenizer from a different provider, plus a rough envelope allowance, gets you there. What matters more is estimating before you build. A system returning long prose and one returning a structured summary can differ by an order of magnitude, and that is a design decision rather than a tuning problem. Then stop estimating as soon as you can measure. Every response carries a usage block with exactly what you were billed. Log it from your first real call, and the whole question becomes arithmetic on data you already have.
Discussion