AI Cost Optimization: A Practical Playbook

Most teams find out their LLM bill has a problem the same way: it shows up on an invoice three times larger than expected, and nobody can point to the one thing that caused it. That is because it is rarely one thing. Model choice, prompt structure, caching, output length, and request volume all compound against each other, and optimizing one in isolation barely moves the number. This is a practical rundown of the levers that actually matter, roughly in the order they tend to pay off, along with where each one stops working and starts creating a different problem.

Why this is worth doing now

Global spending on AI is not slowing down, and neither is the pressure to show it was worth it. Gartner's 2026 forecast puts global AI spending at $2.59 trillion for the year, a 47% increase over 2025.

"Global spending on AI will rise by 47% year-over-year in 2026, totaling $2.59 trillion."

CIO Dive, reporting Gartner's 2026 AI spending forecast

That figure covers the whole market, chips and cloud infrastructure included, not just API bills. But the enterprise-specific number tells the same story from a narrower angle: enterprise AI spending is projected to rise more than a third year over year as more of it moves from pilot projects into production traffic that runs every day, all day, whether or not anyone is watching what it costs. The gap between "AI spend is rising" and "AI spend is being managed" is exactly what a cost optimization pass is for.

Lever one: route to the cheapest model that can actually do the job

This is the biggest lever for most workloads, and it is the biggest by a wide margin. Model pricing spans an enormous range, frontier models can cost tens to well over a hundred times more per million tokens than smaller models from the same provider, so the question "does this specific request need the expensive model" is worth more than almost any other optimization you could make.

The catch is that answering that question well is genuinely hard. A router that guesses from surface features of a prompt, length, keywords, apparent complexity, will get some fraction of those guesses wrong, and a wrong guess routed down with nothing checking the output is a bad answer that reaches someone with nobody having caught it. That is not a hypothetical risk; it is the entire reason routing and verification are usually discussed as a pair rather than routing alone. A cheaper model that answers wrong and ships anyway did not save you money, it moved the cost downstream to whoever has to notice and fix it.

llm11 takes the position that routing without a check is routing on faith: a triage layer picks a candidate model per request, then a verification pass runs before the answer goes out, escalating to a stronger model automatically when a check fails. That is the mechanism behind llm11's cost-first positioning: the savings are real because the cheap answers that would have been wrong get caught and retried rather than quietly shipped.

Lever two: prompt caching, and its real interaction with routing

Several providers now discount repeated calls that reuse the same prompt prefix against the same model, often steeply, which makes a stable system prompt or a large document held in context much cheaper on the second and subsequent calls than the first. This is a large lever on its own for workloads with a long, unchanging prefix.

Where it gets complicated is the intersection with routing, and it deserves more depth than a paragraph here can give it. The short version: a cache is generally keyed to the specific model you called, so routing a request to a different, cheaper model breaks a warm cache and can make the "cheaper" route a net loss once you count the lost discount on the prefix. This site's dedicated post on the caching and routing interaction walks through exactly when that tradeoff bites and why it is a harder problem to solve than it sounds, so treat this section as a pointer rather than a repeat.

Levers three and four: batching, and output length control

If your workload has any tolerance for latency, batch APIs are close to a free discount. Anthropic's Batch API, for example, applies a 50% discount on both input and output tokens for asynchronous processing, according to Anthropic's own pricing documentation. That is not a routing decision or an architectural change, it is a different API mode for requests that do not need an answer in the next second. The obvious limit is that batching only works for workloads that can tolerate the delay: a batch job processing yesterday's support tickets overnight is a perfect fit, a live chat response is not, no matter how much the discount tempts you to try.

Output tokens are usually priced well above input tokens, often five times higher or more depending on the model, so an unbounded or verbose response costs disproportionately more than the same request answered tightly. Setting explicit output limits, asking for structured or terse responses where the use case allows it, and trimming few-shot examples and boilerplate instructions in the prompt itself are cheap changes that compound across every request, not just the expensive ones. This lever has a ceiling too: cut a response short enough and you start losing the parts of the answer that mattered, which is a quality regression dressed up as a cost win. The goal is trimming waste, not trimming substance.

Lever five: semantic caching for repeated queries

Separate from provider-side prompt caching, semantic caching stores the answer to a previous query and serves it again when a new query is close enough in meaning, using an embedding similarity check rather than an exact string match. For workloads with genuinely repeated questions, a customer support FAQ, a documentation search bar, a set of common analytical queries, this can eliminate a meaningful share of model calls entirely.

The honest limit here is the word "repeated." Semantic caching only helps when a real chunk of your traffic is asking variations of the same thing. Point it at a workload of genuinely novel, one-off requests and it does nothing, and worse, a badly tuned similarity threshold can serve a stale or wrong cached answer to a query that only looked similar. It is a targeted tool for a specific traffic shape, not a general-purpose discount.

Putting the levers in order, and how they interact

Impact ranking is a general pattern observed across routing and pricing literature, not a guaranteed order for every workload; the right sequence depends on your own traffic mix.

None of these five levers operate independently, which is the part a checklist approach misses. Route a request to a cheaper model and you may lose a prompt cache hit on the same call. Cache aggressively and you may mask how much a bloated prompt is actually costing you before caching hides it. Batch a workload and you lose the option to route it live if conditions change mid-run. The right sequence for most teams is to fix the biggest, most independent lever first, usually routing, get a verification layer in place so routing down does not become a quality liability, and only then layer in caching and batching for the specific traffic shapes that benefit from them. Doing it in the reverse order tends to produce a system where each optimization gets partial credit for savings that a different lever already claimed, and nobody can tell you the real number.

What "cheap" actually means before you optimize for it

It is worth being precise about a term this whole playbook leans on. "Cheap" can mean the lowest list price on an individual model, the lowest blended cost across a mixed workload, or the lowest total cost once you count the price of wrong answers that had to be redone. Those three are not the same number, and optimizing for the first while ignoring the third is how a routing project ends up looking successful on a spreadsheet and worse in practice. This site's deeper look at what "cheapest" actually means works through that distinction with real per-model pricing and a worked blended-cost example, and it is worth reading before you commit to any specific routing strategy.

Frequently asked questions

What is the single biggest lever for AI cost optimization?

For most production workloads it is model selection: routing requests that do not need your strongest model to a cheaper one that can still handle them. The gap between frontier and smaller model pricing is large enough that this typically dwarfs every other optimization, provided the routing decision is checked rather than left to guesswork.

Does prompt caching conflict with model routing?

Yes, in a specific and easy-to-miss way. Provider-side prompt caches are generally tied to the model you called, so routing a repeat request to a different, cheaper model starts that cache cold again. Depending on how large the cached prefix is, the lost discount can outweigh the price difference between models, which is worth understanding in more depth before you route caching-heavy traffic aggressively.

Is AI cost optimization just about picking cheaper models?

No. Model selection is usually the largest single lever, but caching, batching, output length control, and semantic caching for repeated queries all address different parts of the bill, and a mature FinOps practice for AI spend tracks all of them together rather than optimizing one and assuming the job is done.

How do I know if my cost optimization is actually working?

You need visibility into what you are spending and why, broken down by request type, model, and outcome, not just a total. Once the levers above are in place, tooling built for tracking LLM spend and behavior over time is what tells you whether the changes held up in production or quietly regressed a month later.

Does llm11 charge extra for cost optimization features?

No. Billing on llm11 is prepaid credits, spent at provider list price, with a percentage fee applied only when you buy credits, not on every token or feature. The pricing page has the exact mechanism and current figures.