Token & Cost Optimization for AI Development
Most LLM bills are 5–10× larger than they need to be. The fixes are structural — index instead of dump, cache instead of resend, route instead of defaulting to the flagship model — and each one is measurable.
Last reviewed Jul 7, 2026
What tokens cost right now
Prices pulled from our AI tools database at build time — as of 2026-07-28. Example workload: 2,500 input + 400 output tokens per request,100,000 requests/month.
| Model | Input / 1M | Output / 1M | Example monthly cost |
|---|
| Grok 4 Fast · Grok | $0.20 | $0.50 | $70 |
| DeepSeek-V3.2 · DeepSeek | $0.28 | $0.42 | $87 |
| Gemini 2.5 Flash · Gemini | $0.30 | $2.50 | $175 |
| Mistral Medium 3 · Le Chat (Mistral) | $0.40 | $2.00 | $180 |
| Claude Haiku 4.5 · Claude | $1.00 | $5.00 | $450 |
| Sonar · Perplexity | $1.00 | $1.00 | $290 |
| Claude Sonnet 5 · Claude Introductory pricing through Aug 31, 2026; then $3 in / $15 out. | $2.00 | $10.00 | $900 |
| Gemini 3 Pro · Gemini | $2.00 | $12.00 | $980 |
| Mistral Large · Le Chat (Mistral) | $2.00 | $6.00 | $740 |
| GPT-5.4 · ChatGPT | $2.50 | $15.00 | $1,225 |
| Grok 4 · Grok | $3.00 | $15.00 | $1,350 |
| Sonar Pro · Perplexity | $3.00 | $15.00 | $1,350 |
| GPT-5.5 · ChatGPT | $5.00 | $30.00 | $2,450 |
| Claude Opus 4.8 · Claude | $5.00 | $25.00 | $2,250 |
The spread between the cheapest and the most expensive model on this exact workload is $2,380/month — which is why routing requests to the right model tier is usually the single biggest saving.
1. Send an index, not the codebase
Do: Build a structural index (code knowledge graph, embeddings, or symbol map) of your repo or corpus, and hand the model only the callers, dependencies, and snippets relevant to the change. Tools like code-review-graph do this incrementally on every commit.
Don't: Paste whole files — or worse, several files 'for context' — into every prompt. A single review request with full files can burn 10–15k tokens where 100 would do.
Why: This is the single biggest lever. On this very website's repo, a pre-commit code review that would have needed 13,652 tokens of raw file context ran on 70 tokens of graph context — a 99% reduction, on every single commit.
2. Cache the static half of every prompt
Do: Structure prompts so the stable prefix (system instructions, tool schemas, style guides, few-shot examples) is byte-identical across calls, then enable prompt caching. Put volatile content (user input, retrieved chunks) at the end.
Don't: Interpolate timestamps, request IDs, or user names into the system prompt — one changed byte at the top invalidates the whole cached prefix.
Why: Cached input tokens cost ~10% of normal on Claude and OpenAI models. An agent with a 3,000-token system prompt called 100k times/month saves real money for a one-day refactor.
3. Route requests to the right model tier
Do: Classify each request (heuristics or a tiny classifier) and send mechanical work — extraction, formatting, classification, simple Q&A — to a small model. Escalate to the flagship only on complexity or failure.
Don't: Default every call to the flagship because 'quality'. Most production traffic is mechanical.
Why: Per-1M-token prices differ 10–25× between tiers (see the live table above). If 70% of traffic runs on the small tier, the blended bill drops by more than half with no visible quality loss.
4. Treat output tokens as the expensive ones
Do: Set max_tokens per call type, request structured JSON with only needed fields, and ask for diffs or deltas when the model edits something.
Don't: Let the model restate the input, echo entire documents back, or produce prose when a boolean and a reason would do.
Why: Output is priced 3–6× input on most current models — $25/1M out vs $5/1M in on Claude Opus 4.8. Verbose outputs also compound: they re-enter context on the next turn.
5. Batch everything that isn't interactive
Do: Move embedding generation, backfills, nightly summarization, evals, and content pipelines to the provider's batch API with a 24h window.
Don't: Run offline jobs through the synchronous API at full price.
Why: Batch pricing is typically 50% of synchronous pricing for identical results — the easiest discount in the industry to claim.
6. Prune and summarize conversation history
Do: Keep the last few turns verbatim, replace older turns with a rolling summary, and drop stale tool results entirely once they've been acted on.
Don't: Resend the full transcript every turn. A 40-turn agent session resending everything pays for turn 1 forty times.
Why: Context grows quadratically in cost across a session (each old token is re-billed every new turn). History pruning turns that quadratic into roughly linear.
7. Make token spend visible per feature, in CI
Do: Log input/output/cached token counts tagged by feature and model on every call; fail CI or alert when a feature's per-request tokens regress past a budget.
Don't: Discover regressions in next month's invoice, where they're aggregated beyond diagnosis.
Why: Teams that see tokens per PR treat them like bundle size and keep them flat; teams that see only monthly bills drift upward ~10-20% per quarter.
The real-world case study behind this playbook
This playbook exists because we hit the problem ourselves, building this site. AI-assisted development kept re-reading the same files: every code review or change request began with the assistant grepping and reading thousands of lines to rebuild context it had already had yesterday.
The fix was structural, not clever prompting. We indexed the repository with code-review-graph, which parses the codebase with Tree-sitter into a persistent graph of files, functions, callers, and imports, updated incrementally by a pre-commit hook. When a review is needed, the assistant queries the graph for exactly the affected functions and their blast radius instead of reading files.
The measured result on a recent commit to this repository: full-file context would have cost 13,652 tokens; graph context cost 70 tokens — a 99% reduction, repeated on every commit, forever. The same pattern applies to any corpus: index once, query small.
Where the money actually goes
Across production LLM apps, spend concentrates in four places: repeated static prompts (fixed by caching), oversized context (fixed by indexing/retrieval), flagship-by-default routing (fixed by tiering), and verbose output (fixed by output discipline). Work them in that order — caching and context are usually one-day changes with the largest immediate effect.
The live table above shows current per-1M-token prices from our tools database. Prices move often — Anthropic and OpenAI both repriced flagship tiers in 2026 — which is exactly why this page is rebuilt several times a day from the same data that powers our AI tools comparison, and why our daily freshness job watches both vendors’ pricing pages for changes that would invalidate the math here.