Controlling AI costs: model routing, prompt caching and batching

An AI feature that costs a few dollars in a pilot can cost real money at production volume, and the bill is driven by three things you control: which model handles each request, how much of each request is repeated text you pay for again, and whether the work needs an answer right now. This explainer covers the three standard levers, using one vendor's published list prices as a concrete example. Other providers offer equivalents, but the numbers below are Anthropic's, as of this week.

Lever 1: route by task, not by habit

Using the most capable model for everything is the most common source of overspend. A document classifier, a field extractor and a contract-reasoning step do not need the same model. Routing means sending each task type to the cheapest model that passes your own accuracy tests, and escalating only when it does not.

List prices show why this matters. On Anthropic's pricing page, Claude Opus 4.8 is $5 per million input tokens and $25 per million output tokens ^[2]. Claude Fable 5, released June 9, is $10 and $50 ^[1][2]. Anthropic describes Fable 5 as less than half the price of its earlier Mythos Preview ^[1].

Routing is also appearing inside products. Fable 5 uses classifiers to detect cybersecurity, biology/chemistry and distillation requests and routes them to Opus 4.8, notifying the user; Anthropic says this fallback triggers in under 5% of sessions on average and is deliberately conservative, so some benign requests are caught ^[1]. That is a safety design rather than a cost design, but it shows the pattern: a classifier up front, a model choice behind it.

A practical routing setup has three parts: a small set of task types, a test set of real examples per type, and a rule for escalation when a cheaper model's answer fails validation.

Lever 2: cache the text you keep resending

Most business prompts begin with the same material: instructions, policies, a product catalog, a long document many questions will be asked about. Without caching, you pay full input price for that prefix on every call.

Anthropic's prompt caching bills in three tiers relative to the base input price: writing a five-minute cache costs 1.25x, writing a one-hour cache costs 2x, and reading from the cache costs 0.1x ^[3]. The cache refreshes at no cost each time it is used, and a hit requires an exact match of the cached prefix ^[3]. Prompts shorter than the model's minimum (1,024 tokens for Opus 4.8, 512 for Fable 5) are simply processed without caching ^[3].

An illustration, using the published Opus 4.8 prices: a 10,000-token policy document attached to every request costs $0.05 per call uncached ($5 per million tokens) and $0.005 per call when read from cache. The first call pays a one-time write of $0.0625 (5-minute cache). If the same prefix is reused even twice within the cache window, caching is already cheaper than not caching. This arithmetic is ours, derived from the list prices ^[2]; real savings depend on how often calls arrive inside the cache window.

Caching only helps where the repeated text is at the start of the prompt and is byte-identical. Putting a timestamp or user name before the policy text defeats it.

Lever 3: batch what can wait

Anthropic's Batch API processes large volumes of requests asynchronously at a 50% discount on both input and output tokens, with Opus 4.8 listed at $2.50 / $12.50 per million tokens in batch mode ^[2]. Anthropic's documentation says batch and caching discounts can be combined ^[2]. Good candidates are overnight document classification, back-catalog summarization and evaluation runs. Anything a person is waiting on is not.

How to start

1. Log token usage per feature, split into input, output and cached.

2. Pick the one feature with the highest spend and test a cheaper model on 100 or more real examples.

3. Move static instructions to the front of the prompt and enable caching.

4. Move any job without a live user to batch.

5. Re-check list prices monthly: they change, and new releases such as Opus 4.8 (May 28) kept the prior price tier ^[4].

Opus 4.8 launched at $5 / $25, unchanged from Opus 4.7 ^[4], which is a reminder that the biggest savings usually come from architecture, not from waiting for prices to fall.

For most teams, the order of operations is measure first, then cache, then route, then batch. Each step is reversible and can be tested against your own accuracy bar before it reaches users.