How to Use Expensive AI Models Without Burning Through Your Credits

A familiar pattern plays out for anyone building with frontier models. The most demanding parts of a project, such as a content agent system, go to the most capable model available, usage climbs faster than expected, and a little arithmetic shows a month’s allowance disappearing in about a week.

The more useful discovery comes when you trace where the tokens actually went. Very little of the spend goes on the hard questions. Most of it goes on the way the questions are asked.

Every frontier model bills in the same handful of expensive ways, and once you have seen the pattern you will recognise it in whichever model replaces today’s next year. This article uses Anthropic’s current top model, Claude Fable 5.1, as the running example because its prices make the stakes obvious: $10 per million tokens you send in, $50 per million it sends back, and $0.25 per million for input it can read from its cache (Anthropic pricing), although none of what follows depends on that model.

You are paying for your chat history, not your question

The first thing that changes how you should use a metered model is this: every message you send carries the entire conversation with it. The model keeps no memory between messages, so the whole history is bundled in as input every time. Your first message in a chat might be 500 tokens. By message thirty, a question of the same size is riding on top of 50,000 tokens of history, and that history is processed again with every reply.

That makes the single long chat you keep alive for days, because it “has all my context”, the most expensive way to use the model. Each new message costs more than the last, even when the questions get simpler.

Providers now soften this with caching, covered in more detail below, because a cached history is read at a fraction of the price. But a cache expires after minutes rather than hours without use, so a conversation you return to after lunch is often billed at full price again (the section on prompt layout below covers how long caches last in each product). The habit that holds up is simple: start fresh chats deliberately. When a task is done, the chat is done, and if you need to carry context forward, paste in the two paragraphs that matter instead of the whole transcript.

Files make this worse in a way that is easy to miss. Upload a 100-page PDF, ask a question, and the whole document comes along as input for every follow-up in that chat. Ten follow-ups means paying for the document ten times. Pull out the section you need and ask about it in a new chat.

Then there is the ratio that should change how you prompt. Across Anthropic’s current lineup, output costs five times as much as input, from Claude Haiku 4.5 at $1 and $5 to Claude Fable 5.1 at $10 and $50. Most people work hard at trimming their prompts, which saves money at the cheaper rate, while the model writes 800-word answers at the expensive one. A single line such as “keep answers short unless I ask for detail” is a budget instruction, and it is worth more than anything you trim from your own prompt.

Two further habits pay for themselves straight away, and the first is that when an answer comes back wrong, you should not reply “no, try again” or press regenerate, because both re-send the full context for another full-price attempt; instead, edit your original message to be more specific, which fixes the cause rather than paying again for the symptom. The second is to add one line before any large task: “ask me clarifying questions before you start.” A vague request to a capable model produces a long, confident, wrong answer at the output rate, followed by rounds of correction. Being interviewed first turns three paid drafts into one.

Turn the effort down before you change the model

Most current models let you choose how hard they think, and the instinct is to set that to maximum because it feels safer, but the published measurements suggest otherwise. In Anthropic’s own benchmark runs on research and knowledge work, medium effort matched the accuracy of the default high setting at 70 to 87 percent of the cost, and low effort gave up one to three points for a third to a half off. On a demanding coding benchmark, Claude Opus 5.5 at medium effort scored about 2.5 points below high for roughly 70 percent of the cost (Anthropic, optimizing for cost and intelligence).

The more surprising finding concerns which model to use at all. On that same coding benchmark, Claude Fable 5.1 at low effort solved 88.6 percent of tasks at $0.54 per solved task, while Claude Sonnet 5, a cheaper mid-tier model, solved 77.4 percent at $0.84. The model with five times the per-token price was 35 percent cheaper per finished result, because it needed fewer attempts and wrote less to get there.

This is the most common mistake in cost-cutting: judging a model by its price per token instead of its cost per completed task. A cheap model that needs three tries is not cheap. Before building anything elaborate, try the capable model at a lower effort on a sample of your real work, and compare what each approach costs per task that actually finishes correctly.

Use the expensive model to grade, not to generate

Once you have that baseline, the next saving comes from the five-to-one price gap between output and input. The capable model is often more valuable as a reviewer than as a writer. When it generates something, you pay the premium rate for every word of a long output. When it reviews something, it reads a great deal at the cheaper input rate and writes a short verdict. You get the same judgment for a fraction of the cost.

The practical move is to reverse the pipeline: let a cheaper model produce the draft, the code or the analysis, then give that output and the original goal to the capable model with one question: “what is wrong with this, and what specifically would you change?” Set up this way, a cheaper model handles the volume and the capable model handles quality control at either end.

The next step stops you paying for the same judgment twice. When the capable model catches a mistake, do not stop at fixing it. Show it the bad output and ask it to write the rule that would have prevented it, then save that rule where the cheaper model reads it at the start of every session. Rules written from real failures are far better than rules written in the abstract. Ask a model cold for “a skill file about careful coding” and you will get generic advice; show it an actual failure and the rule comes back specific, checkable and aimed at a mistake your setup really makes.

A prompt that works well, pasted together with the failed output:

Here is a task I gave a cheaper model, and here is the output it produced. The specific mistake was: [describe it in one sentence]. Write a SKILL.md file that would have prevented this exact mistake. Describe the discipline as rules about your own behaviour, not instructions to someone else. Every rule must be checkable, meaning I could look at an output and say yes or no, it followed this. Keep it under 300 words. Do not include general best practices that don’t relate to this specific failure.

The last sentence does most of the work, because without it you get padding, and padding in a rule file actively hurts, because every extra instruction competes for the model’s attention with the ones that matter.

Test every rule before you trust it, because an untested rule can quietly make results worse while you assume it is helping. Take three to five small tasks of the kind the rule targets, since a single run is noise. Run each one twice on the cheaper model, with and without the rule, label the pairs A and B at random, and have a third model grade them without knowing which is which. If the version with the rule wins most pairs, keep it. If the results split evenly, the rule is doing nothing, and if it loses, remove it.

One more asset is worth having the capable model build while you have its attention: the plan. A detailed specification is what makes a cheaper model look capable. Given a vague request, a mid-tier model spends tokens guessing what you meant and often guesses wrong. Given a precise plan, with the decisions made, the file structure named and the edge cases listed, it mostly has to execute, which is what cheaper models do well. For anything substantial, a fixed split works well: the capable model writes the plan, the cheaper model builds from it, and the capable model reviews the result, so the expensive model stays out of the middle.

Five ways to turn rented judgment into something you own

Everything above makes your own work cheaper, but the harder goal is a system that keeps producing good work after the expensive session ends. Each time you ask a frontier model a question, you are renting its judgment for that moment. The five conversions below turn that rented judgment into assets you keep: tests, a ladder, isolated workers, cache-stable prompts and curated memory.

1. Judgment into tests

Most instructions to a model are judgment calls in disguise: “write clean, well-tested code”, “handle edge cases well”, “keep the tone professional”. A cheaper model cannot check its own work against a judgment call, so it produces something, you find the problems, and the same kind of mistake returns next week because nothing about the fix was saved where the model could check against it.

The shift is to have the capable model write a test that verifies “good” automatically. A rule asks the model to be careful, whereas a test asks nothing and simply checks, pass or fail, every time, at no cost.

I’m building [describe the specific function, feature or piece of content]. Before any code gets written, write the acceptance tests: the concrete, checkable conditions that define whether this is done correctly. For each test, state the exact input, the exact expected output, and why this case matters. Include the obvious cases and at least three edge cases most people would miss. Do not write the implementation. Only write the tests.

Suppose you are building a function that processes refund requests for an online shop. “Handle edge cases well” gets you code that works for an ordinary refund and silently fails on cases you did not think to mention. The prompt above gets you tests like these instead: a refund requested exactly on the deadline is approved, because off-by-one errors on date boundaries are among the most common refund bugs; a refund one day late is rejected with a clear reason code; a negative amount is rejected before any processing happens; an order that has already been refunded cannot be refunded again; and a missing or malformed customer ID is rejected at the input layer. The cheaper model then receives one instruction: implement the function and keep working until all five tests pass.

Two lines in that prompt matter more than they appear to. “Do not write the implementation” stops the capable model from writing the whole function at premium output rates, when what you want from it is a short, permanent artefact. And asking for the reason behind each test means that when one fails six months later, you can tell whether the code is wrong or the requirement has changed.

The same move works for written work, where the test becomes a checklist that a model with no context could apply. Ask for items that can each be answered yes or no from the draft alone, covering structure, evidence and voice separately, and a cheaper model can grade every draft against them while the capable model never reviews another draft directly.

2. Model selection into a ladder

Without a system, you decide on every task whether it feels important enough for the expensive model. That makes you the routing system, and not a reliable one, because the real difficulty of a task is often invisible until it has already failed.

With a ladder, the cheaper model attempts everything first and checks its output against the tests from the first conversion. Only a genuine failure moves up to the capable model, so cost follows actual difficulty rather than your anxiety about which tasks feel important. The core of it is the self-check instruction given after each task:

Here is the task you just completed: [task]. Here are the acceptance tests: [tests]. Check your own output against each test individually and state pass or fail for each one, with your reasoning. If any test fails, do not attempt to fix it. Just report the failure clearly so it can be escalated.

“Do not attempt to fix it” matters because a cheaper model asked to repair its own failure will often argue that the test does not really apply, which defeats the point of an objective check.

Imagine a pipeline producing 50 product descriptions a day, each checked against five tests: length within range, required keywords present, no banned claims, tone consistent with the brand, and no contradiction with the product specification. If 44 pass and 6 fail, only those 6 reach the capable model, each with its specific failure attached, so it starts from a named problem rather than from scratch.

The measurements support this pattern with one refinement: in Anthropic’s coding runs, running every task at low effort and re-running only the failures at high effort reached about 97 percent at roughly $0.17 per task, compared with 95.3 percent at $0.29 for running everything at high effort. The ladder does not have to climb to a different model; climbing to a higher effort on the same model often captures most of the saving, and keeps a single cache.

3. One shared context into isolated ones

The default for many agents is a single, continuously growing context that reads files, writes code, checks results and decides what to do next. Because every turn re-sends the whole history, the cost compounds quietly in an automated system where nobody watches the conversation getting longer.

The alternative is an orchestrator that holds almost nothing, just the plan and a status list, and hands each piece of work to a fresh worker that receives only what it needs:

You are managing a multi-step task. Keep only two things in your own context: the overall plan, and a status list showing what is done, in progress or pending. For each piece of work, delegate to a fresh subagent with only the information it needs to complete that one piece. When a subagent reports back, record only the short result in your status list and discard its context. Never accumulate a subagent’s full working history in your own context.

Suppose an agent is auditing 15 files for a particular pattern. Without isolation, by file 10 every instruction is re-sending nine files’ worth of history that has nothing to do with file 10. With isolation, the orchestrator’s context reads “files 1 to 9 audited, no issues; file 10 in progress; 11 to 15 pending”, and the worker for file 10 sees only file 10 and the pattern to look for.

The research adds two cautions. First, splitting work across agents pays off mainly when there is genuinely too much for one context. In Anthropic’s measurements, a coordinator with 25 cheaper workers cut the cost of a corpus too large for any single context window roughly in half, but scored 10 to 12 points lower, and on work that fit in one context the single model at lower effort came out ahead. Second, for long single-agent runs, the provider’s own context tools can do part of this job: automatic compaction, which summarises older turns, saved 32 percent on a long agent run in the same tests.

The same instinct, less shared context, produces a better verifier. If a second agent checks the first agent’s work, show it only the specification and the result:

Here is the original specification: [spec]. Here is the final output to check: [output]. Evaluate whether the output meets the specification. Do not ask for or consider any explanation of how or why the output was produced. Judge only the specification against the result.

A verifier that reads the builder’s reasoning can be talked into approving something wrong by a well-written justification, the same way a persuasive explanation in a code review can make a bug look intentional. One that sees only the specification and the output cannot be persuaded.

4. Prompt layout into a pricing decision

Prompt caching is now the largest single cost lever available. In Anthropic’s published measurements it cut the cost of agent loops by a factor of 2.7 to 5.3, and agent traffic read a median 84 percent of its input from cache over a day. Every major provider offers it in some form: Anthropic bills cache reads at a tenth of the input price on most models, a twentieth on Claude Opus 5.5 and a fortieth on Claude Fable 5.1; OpenAI caches prompts of 1,024 tokens or more automatically on its newest models and bills reads at a tenth of the input price (OpenAI prompt caching); and Google applies implicit caching by default on Gemini 2.5 and newer models (Gemini context caching).

Caching only works on an identical prefix, matched byte for byte from the start of the prompt. That turns the physical order of your prompt into a pricing decision. Many prompts weave the standing rules and the day’s task together, which reads naturally but means nothing at the front of the prompt repeats exactly between calls. The fix is a template: an identical static block first, containing your rules, reference material and standing instructions, followed by a variable block with today’s task and data. Never let variable content appear before the static block, and never change the static block’s wording or order, because a single reordered sentence breaks the match.

To see the arithmetic, take a 4,000-token static block, a 200-token daily task and 1,000 calls. Woven together, nothing matches, and you pay full price for 4.2 million input tokens. With the static block first, you pay a small premium to write it to the cache once (1.25 times the input price for Anthropic’s five-minute cache), then read it at the cached rate on every later call. That comes to roughly 605,000 full-price-equivalent tokens on a model with the standard tenth-price cache reads, which is about seven times cheaper, and roughly 305,000 on Claude Fable 5.1, which is about fourteen times cheaper, and the only difference is where the words sit.

Caching also stacks with batch processing: if nobody is waiting on the result, Anthropic’s Batch API takes 50 percent off every token, cached ones included, in exchange for results arriving within 24 hours.

How long a cache lasts, and what a break costs

The catch with caching is that it expires, and each product handles that differently. When a cache has expired, the next message does not just lose the discount: the whole prefix, including the entire conversation so far, is processed again at full price, and on Anthropic’s API it is written back to the cache at a premium. A long conversation picked up after a break is therefore one of the most expensive single messages you can send.

Where you use the model	How long the cache lasts	What to do about it
Anthropic API	Five minutes by default, renewed free each time it is used. A one-hour option costs twice the input price to write instead of 1.25 times.	Use the one-hour option when requests arrive more than five minutes but less than an hour apart. The clock starts when a request begins, so a response that takes four minutes to stream leaves about one minute to reuse the cache.
Claude Code	One hour on a Pro, Max, Team or Enterprise subscription, dropping to five minutes once you draw on usage credits. Five minutes by default with an API key or a cloud provider.	Use /clear between unrelated tasks; it costs nothing. /usage shows whether the cache is warm or cold. After a long break, resume from a summary when offered rather than reloading the full history, and remember that /compact is itself a large request.
OpenAI API	Automatic for prompts of 1,024 tokens or more on GPT-5.6 and later, kept for 30 minutes, with cached reads at a tenth of the input price. Older models keep entries for roughly 5 to 10 minutes, or up to 24 hours with extended retention.	On older models, send a stable prompt_cache_key with requests that share a prefix so they reach the same cache.
Gemini API	Implicit caching is automatic on Gemini 2.5 and later for prompts above 2,048 to 4,096 tokens, depending on the model, with cached input at about a tenth of the price. Explicit caches live as long as you set them to.	Explicit caches are billed for storage by the hour, for example $1.00 per million tokens per hour on Gemini 3.5 Flash, so delete one when the work is done rather than leaving it to run up storage.
Consumer chat apps	Providers do not publish cache lifetimes for their chat apps.	Treat a conversation left idle as expensive to resume. Start a fresh chat and paste in the context that matters.

Sources: Anthropic prompt caching, Claude Code costs, OpenAI prompt caching, Gemini context caching and Gemini pricing.

One more cause of silent cache misses is worth knowing: changing anything near the front of the prompt resets everything after it. On Anthropic’s API, editing the tool definitions clears the whole cache, and adding an image or changing the effort setting clears the cached conversation. Settle those choices at the start of a session rather than adjusting them midway.

5. Accumulated notes into curated memory

The last conversion concerns the folder of lesson files your models read before they work. After a few months it usually contains contradictions nobody resolved, duplicate lessons written in different words, and at least one fix that was once correct and is now wrong because what it addressed has changed. The model reads all of it with equal trust.

A scheduled consolidation pass fixes this, and it suits the capable model because it is high-judgment work with very little output:

Here is my full notes folder: [contents or file list]. Do four things. First, find and merge any notes that say the same thing in different words. Second, find any notes that contradict each other, and for each contradiction state which one is correct and why, then mark the other as superseded. Third, flag any note that refers to a tool, process or fact that may be outdated, and ask me to confirm before deleting it. Fourth, identify any lesson that appears in three or more separate notes in different forms, and draft it as a permanent rule suitable for a SKILL.md file.

Consider three notes written months apart: one reminding you to double-check date formats from European sources, one recording another mix-up between day-month and month-day order, and one saying the bug was fixed by hard-coding the US format, a fix that broke when an international client arrived later. A consolidation pass recognises the same recurring problem, flags the third note as harmful, and promotes a permanent rule: when the source data includes a country or locale, parse dates using that locale’s format, and never assume a format from convention alone.

Weekly tends to be the right rhythm, because after every session there is rarely enough new material to find patterns, and monthly lets contradictions sit live for weeks.

Putting it together

None of this needs to be built in a day, and the order matters. Turn on caching and fix your prompt layout first, because it is the largest saving and takes minutes. Then measure the capable model at a lower effort against your real work, and judge everything by cost per completed task. Write tests next, since the ladder depends on having something concrete to fail against. Isolate contexts once you are running real multi-step agents, and only where the work is too large for one context. Curate memory once you have enough notes for patterns to exist, usually after a few weeks.

A month’s allowance disappearing in a week is rarely a problem with the model or its price. In a well-run system, the expensive model spends its time at the edges of the work: writing tests and plans, setting up cache-stable rules, reviewing results and running the weekly clean-up, all short and all reusable. Whether the next frontier model costs $50 per million output tokens or twice that matters much less once you are no longer renting its intelligence by the hour, because most of it has already been turned into things you own.