LLM API Costs Explained: What You Pay For and How to Cut the Bill
LLM API costs come from tokens: you pay a per-million-token rate for the text you send (input) and a higher rate for the text the model generates (output), plus extra for some tools like web search. Your bill is roughly requests × tokens per request × price per token. You control it by choosing the smallest model that does the job, keeping prompts and outputs short, caching repeated content, batching non-urgent work, and setting hard spend limits.
Key takeaways
- Price varies by model tier more than by provider; picking the right tier is the biggest lever.
- Output tokens cost several times more than input tokens on current models.
- Prompt caching and batch processing can cut costs substantially for the right workloads.
- Retrieved documents, conversation history and tool definitions all add input tokens on every call.
- Set provider spend alerts and hard limits before launch, not after the first surprise invoice.
How LLM pricing works
Every major provider charges per million tokens (MTok), split into:
- Input tokens: your system prompt, instructions, retrieved documents, conversation history, tool definitions and the user's message.
- Output tokens: everything the model writes, including any visible reasoning or tool-call arguments.
- Cached input tokens: repeated prompt prefixes billed at a discount.
- Tool fees: some built-in tools have per-call charges.
Token counts depend on the tokenizer. Anthropic states that Claude 4.7 and later models use a newer tokenizer that produces about 30% more tokens for the same text than earlier models (Anthropic pricing). Always measure with the specific model you plan to use.
Current price points (as of October 2026)
| Provider and model | Input / MTok | Output / MTok | Cached input / MTok |
|---|---|---|---|
| OpenAI gpt-6-luna | $0.10 | $0.50 | $0.01 |
| OpenAI gpt-6.1-terra | $1.00 | $5.00 | $0.05 |
| OpenAI gpt-6.1-sol | $2.00 | $10.00 | $0.10 |
| OpenAI gpt-6-astra | $10.00 | $50.00 | $1.00 |
| Anthropic Claude Haiku 5.5 (prompts ≤100K tokens) | $0.10 | $0.50 | $0.01 (cache hit) |
| Anthropic Claude Sonnet 5.5 | $2.00 | $10.00 | $0.10 (cache hit) |
| Anthropic Claude Opus 5.5 | $4.00 | $20.00 | $0.20 (cache hit) |
| Anthropic Claude Fable 5.1 | $10.00 | $50.00 | $0.25 (cache hit) |
Sources: OpenAI API pricing (standard tier) and Anthropic pricing. Notes:
- Claude Haiku 5.5 rises to $0.50 input and $2.50 output per MTok for prompts over 100,000 tokens.
- Anthropic charges for writing to the cache: 1.25× base input for a 5-minute cache, 2× for a 1-hour cache.
- Both providers offer batch processing at 50% off standard rates for non-urgent work.
- Regional or US-only data processing adds about 10% on both providers' newer models.
- Web search on both providers is listed at $10 per 1,000 searches, plus the tokens of the content retrieved.
The spread is enormous: the top tier costs 100 times the budget tier per token. Matching model to task matters more than haggling over providers.
A worked example
A support assistant handles 10,000 conversations a month. Each conversation averages four turns. Each turn sends a 1,500-token system prompt, 3,000 tokens of retrieved documents, about 1,000 tokens of conversation history, and gets back a 200-token reply.
Per turn: about 5,500 input tokens and 200 output tokens. Per month: 40,000 turns, so 220 million input tokens and 8 million output tokens.
| Scenario | Input cost | Output cost | Monthly total |
|---|---|---|---|
| Mid tier ($2 / $10), no caching | $440 | $80 | ≈ $520 |
| Mid tier with system prompt cached (1,500 of 5,500 tokens at $0.10) | $326 | $80 | ≈ $406 |
| Budget tier ($0.10 / $0.50), no caching | $22 | $4 | ≈ $26 |
The budget tier is roughly 20 times cheaper. If it answers your questions accurately enough, it wins. If it does not, a router that sends only hard questions to the mid tier often captures most of the savings. (This simplified example ignores cache-write fees and assumes each turn resends history; real numbers depend on your design.)
Hidden cost drivers
- Conversation history. Resending the full chat every turn grows input linearly. Summarize or trim older turns.
- Retrieved context. Five chunks of 800 tokens add 4,000 input tokens per question. Better retrieval means fewer chunks. See RAG explained for business owners.
- Tool definitions. Every tool's name, description and schema is sent as input. Anthropic also adds a tool-use system prompt of a few hundred tokens per request when tools are present.
- Reasoning and verbosity. Long explanations, chain-of-thought and polite filler are output tokens.
- Agent loops. An agent that retries, re-plans or calls tools repeatedly can multiply calls per task. OWASP lists unbounded consumption among the top LLM risks for exactly this reason.
- Retries on errors. Automatic retries without backoff can double usage during an outage.
12 ways to control LLM API costs
- Start with the smallest model that passes your quality tests, and move up only where it fails.
- Route by difficulty. Use a cheap model to classify requests, then send only complex ones to a bigger model.
- Cap output length with
max_tokensand instructions like "answer in under 80 words." - Ask for structured output (JSON fields) instead of prose when software consumes the result.
- Cache stable prefixes. Put the system prompt, tool definitions and shared documents first, variable content last, so caching applies.
- Batch non-urgent work. Nightly summaries, bulk classification and content generation can use batch APIs at half price.
- Trim history. Keep the last few turns verbatim and summarize the rest.
- Retrieve less, retrieve better. Filter by metadata and lower the number of chunks.
- Cache whole answers in your own app for identical, frequent questions (opening hours, return policy).
- Rate-limit users and features so a single user or bug cannot consume your budget.
- Set provider spend controls. OpenAI lets you set organization and project spend limits; with a hard limit, requests return a 429 error once reached, and alerts can notify you earlier. Enforcement is not instantaneous, so leave headroom.
- Measure per feature. Tag requests by feature or customer so you know which part of your product drives cost.
Voice, images and other modalities
Text tokens are only part of the picture once you add other modalities.
- Voice agents are usually priced per minute. OpenAI lists its GPT-Live 1 voice sessions at $0.05 per minute, billed per second, with backend model and tool usage charged separately. Realtime audio models are priced per million audio tokens at much higher rates than text: gpt-realtime-2.1 is listed at $32 input and $64 output per million audio tokens (as of October 2026). Platforms that orchestrate voice pipelines add their own per-minute fees; LiveKit's calculator, for instance, showed about $0.048 per minute for a default phone-call pipeline. See how to build a voice AI receptionist.
- Transcription is often priced per minute of audio; OpenAI lists file transcription models from about $0.003 to $0.006 per minute.
- Images sent to vision models are converted to tokens, so large or numerous images add up. Resize images to the smallest resolution that still works.
- Hosted tools carry their own fees: OpenAI lists file search tool calls at $2.50 per 1,000 calls and file storage above 1 GB at $0.10 per GB per day.
When you budget a feature, list every modality and tool it touches, not just the chat model.
Building a monitoring routine
- Daily: check spend against budget; investigate spikes.
- Weekly: review tokens per request by feature and the share of requests hitting each model tier.
- Monthly: re-run your quality test set on a cheaper model; prices and models change often.
- On every release: compare tokens per request before and after; prompt changes quietly add cost.
From experience: In technical due diligence and fractional CTO work, the cost problems I look for first are architectural, not pricing: full chat histories resent on every turn, large documents stuffed into prompts, and agents without loop limits. Fixing those usually matters more than switching vendors.
Cost control checklist
- [ ] Quality test set exists, so you can safely try cheaper models.
- [ ] Each feature uses the smallest model that passes tests.
- [ ]
max_tokensset on every call. - [ ] Stable prompt content placed first to benefit from caching.
- [ ] Batch API used for non-urgent workloads.
- [ ] Conversation history trimmed or summarized.
- [ ] Retrieval returns only the passages needed.
- [ ] Agents have step limits, timeouts and loop detection.
- [ ] Per-user and per-feature rate limits in place.
- [ ] Provider spend alerts and hard limits configured.
- [ ] Usage tagged by feature and reviewed weekly.
Next steps
- Log input and output tokens for every call in your app for one week.
- Build a 50 to 100 item quality test set and try the next-cheaper model against it.
- Set spend alerts and a hard limit in your provider dashboard today.
- Apply these levers to your AI customer support agent and review AI agent security and guardrails for loop protection.
- Planning a larger AI rollout? A technical due diligence review can spot cost and architecture risks early.
Sources
- OpenAI API pricing: https://developers.openai.com/api/docs/pricing
- OpenAI spend limits: https://developers.openai.com/api/docs/guides/spend-limits
- Anthropic Claude pricing (caching, batch, tokenizer, tool use): https://docs.anthropic.com/en/docs/about-claude/pricing
- OWASP Top 10 for LLM Applications 2025: https://genai.owasp.org/llm-top-10/
- OpenAI Retrieval guide (chunking, storage pricing): https://developers.openai.com/api/docs/guides/retrieval
Frequently asked questions
What is a token in LLM pricing?
A token is a chunk of text the model processes, often a word or part of a word. Providers charge per million tokens, with separate rates for input (what you send) and output (what the model writes). Token counts vary by model tokenizer; Anthropic notes its newer models produce roughly 30% more tokens for the same text than older ones.
Why is output more expensive than input?
Generating text is more compute-intensive than reading it, so providers price output higher, often five times the input rate on current models. That makes verbose responses, long reasoning and unnecessary explanations expensive. Asking for concise, structured output is one of the simplest ways to reduce cost.
How does prompt caching reduce costs?
When the start of your prompt repeats across requests, such as a long system prompt or document, providers can cache it and charge a reduced rate for the repeated portion. Anthropic charges cache reads at a fraction of base input price, and OpenAI lists separate cached-input rates. Put stable content first and variable content last.
How can I stop an LLM bill from running away?
Set spend alerts and hard limits with your provider, cap max output tokens per request, rate-limit users, and monitor usage per feature. OpenAI supports organization and project spend limits that return an error once reached. Also guard against loops in agents, where one bug can trigger thousands of calls.
Is it cheaper to self-host an open-source model?
Sometimes, at high and steady volume. Self-hosting replaces per-token fees with GPU, engineering and operations costs, and you take on uptime, scaling and security. For most small businesses and early products, API pricing on small models is cheaper and simpler. Re-evaluate once usage is large and predictable.