Anthropic’s official Claude Cookbooks repository just shipped a new cost_optimization/cost_optimization.ipynb notebook, and it’s one of the most practically useful things Anthropic’s Applied AI team has published this year. It’s not a marketing checklist — it’s a working notebook that runs a fictional insurance-claims agent through seven cost-cutting levers, in order of how “free” each one is, while measuring pass rate and cost-per-task at every step so you can see exactly what each optimization actually buys you.
If you’re running Claude-based agents in production and your token bill is starting to hurt, this is the checklist to work through. Here’s what it covers and how to apply it to your own agent.
The Core Idea: Optimize the Architecture, Not the Model
The notebook opens with a framing worth internalizing: “the zeitgeist refers to this phenomenon as a shift from ’tokenmaxxing’ to ‘budgetmaxxing.’ Yet, the defensible solution isn’t to downgrade your models or shrink the scope of your AI use cases.” Instead, the recommended approach is to optimize everything around the model before touching the model itself — because swapping to a cheaper model changes your intelligence ceiling, while the other six levers are close to free.
The checklist, in the order Anthropic’s team recommends applying it:
- Baseline first — get the agent working on a capable model, build an eval, measure baseline cost and pass rate
- Prompt caching — reuse tokens across turns instead of reprocessing them
- Input token management — let the model discover context via tools rather than front-loading everything
- Agent-loop efficiency — keep multi-turn context from compounding
- Output token management — constrain generation with tighter specs
- Batch API — defer non-interactive workloads to an async queue for a 50% discount
- Model selection and effort — find the cheapest tier that still clears your quality bar
Notice model selection sits dead last. That’s deliberate — it’s the easiest lever to pull, but it’s also the one most likely to hurt your product if you pull it too early.
Step 1: Build the Eval Before You Optimize Anything
The notebook’s insurance-claims agent example is instructive. Before touching cost, Anthropic’s team establishes a baseline: Opus running at high effort, full tool loop, no caching, no context management, scored against a 10-claim eval set with known-good answers. That baseline lands at 10/10 correct for roughly $0.29 per task.
The lesson: you need both a pass-rate measurement and a cost-per-task measurement before you optimize, because a model that costs more per token but finishes in fewer turns can end up cheaper overall. Cost-per-task, not cost-per-token, is the number that actually matters. The notebook points to Anthropic’s own guide on demystifying evals for AI agents if you need a starting framework — you don’t need an elaborate eval suite, just a handful of representative tasks with known-good answers to catch regressions.
Step 2: Prompt Caching — The Free Win Most Teams Skip
If your system prompt and tool schemas are static across calls (they usually are), you’re likely paying full input price to reprocess them on every single turn. Prompt caching lets Claude reuse an already-processed prefix server-side: cache for 5 minutes at 1.25x the normal input rate, or 1 hour at 2x, and reprocessing those cached tokens afterward costs only 0.1x normal price.
The simplest entry point is automatic caching — pass cache_control={"type": "ephemeral"} on the request and the API places a single breakpoint after the last cacheable block. The notebook flags the most common way teams accidentally break their own cache: dynamic content (timestamps, request IDs, usernames) sitting above the cache breakpoint in the system prompt. Even one byte of drift invalidates the whole prefix. The fix is to keep the static prefix byte-stable and push anything volatile — like a timestamp — down into the user turn instead.
For more granular control, explicit breakpoints let you place up to four cache_control markers yourself, so a persistent upstream layer (like a policy manual) can keep hitting the cache even when a downstream layer (per-request context) changes on every call. This also lets you mix TTLs — a 1-hour cache on persistent content, 5-minute on ephemeral conversation history.
Step 3: Input Token Management — Progressive Disclosure
Rather than loading everything into the system prompt up front, the notebook recommends progressive disclosure: give the model only the context it needs immediately, with everything else retrievable on demand. Concretely, that means:
- Moving rarely-used reference material (like a ~12K-token underwriting manual in the example) behind a dedicated retrieval tool instead of embedding it in every prefix
- Removing redundant tool-prose recaps from the system prompt, since tool schemas are already rendered into the request
- Using the tool search tool with
defer_loading: Trueon tools that aren’t needed every turn, so onlytool_searchplus un-deferred tools render into the prefix by default
For large artifacts, the notebook recommends pre-downscaling images (1280×720 caps most images at ~1,200 tokens, since image tokenization scales with pixel area) and using the Files API with the code execution tool for large datasets — mount a CSV and let Claude run pandas against it in a sandbox rather than pasting the whole thing into context.
Step 4: Agent-Loop Efficiency — Bound the Compounding
In multi-turn agents, intermediate tokens (tool results, thinking blocks) accumulate on every subsequent call. The notebook covers three techniques to bound that growth:
- Context editing with
clear_tool_uses— removes stale tool results above a token threshold, leaving a placeholder so the model still knows the call happened - Compaction — add
{"type": "compact_20260112"}to summarize older turns once the conversation crosses a threshold (default 150K tokens, 50K floor) - Client-side rolling buffers — for finer control than either server option, prune manually at natural boundaries (like completing a subtask) to keep the message array byte-identical between prunes and protect your cache prefix
For genuinely self-contained subtasks, the notebook also recommends spinning off a subagent — a nested messages loop with its own context that absorbs bulky results and hands back a single-line summary, so the parent agent’s context never sees the raw data.
Step 5: Output Token Management
Three levers here: max_tokens as a hard ceiling (a backstop for runaway generations, not a tuning knob — the model never sees this value), prompting for a specific output shape to naturally shrink normal responses, and stop sequences to catch early-exit conditions (register a sentinel string like <CANNOT_REVIEW> as a stop sequence so the model doesn’t burn tokens explaining a failure it’s already signaled).
Step 6: Batch API for Non-Interactive Work
The Batch API processes requests asynchronously within 24 hours at 50% off every token — and prompt caching discounts still stack on top. The natural fit is anything not time-sensitive: nightly evals, initial triage passes, data exploration. Keep anything user-facing or deadline-bound on the synchronous path, since the 24-hour window is an expiry, not an SLA.
Step 7: Model Selection and Effort — Last, Not First
Only after exhausting the above should you touch the model itself. Start by toggling effort on your current model (Opus and Sonnet support low through max; Haiku 4.5 doesn’t take an effort parameter at all) — this scales thinking and tool-use tokens without changing the model’s underlying capability. Only drop a model tier once you’ve confirmed the eval still passes at your current tier’s lowest effort setting.
For workloads with uneven difficulty, the notebook covers two routing patterns: the advisor tool (a cheap driver model consults a more capable advisor only when it can’t confidently proceed) and task decomposition (assign each subtask to the cheapest model that can handle it — planning and judgment usually need a bigger model, execution is often mechanical enough for a cheaper one).
The Catch: Cheap and Slightly Wrong Isn’t a Win
The notebook’s most valuable section is arguably its honest failure case: a fully decomposed, multi-subagent version of the claims agent runs ~90% cheaper than the Opus baseline and clears every escalation case — but it incorrectly denies the same routine claim across trials, because flattening a full policy manual into a condensed “rule card” for a cheaper decider model dropped a specific carve-out clause. The lesson stated directly: “cheap and slightly wrong is still not an optimization at our bar, and that’s the failure mode our eval exists to catch.” Every optimization needs to be validated against the same eval you started with — a config that saves 90% but silently degrades quality on a real category of cases isn’t a win.
Getting Started
The full notebook is available in the official repo, along with a Pareto-frontier chart plotting cost-per-10K-claims against pass rate across every configuration tested. Running the notebook top to bottom costs roughly $40 in API credits per Anthropic’s own estimate, so it’s worth reading through first and running specific cells against your own use case rather than executing the whole thing blind.
Sources
- Claude Cookbooks: cost_optimization/cost_optimization.ipynb — Anthropic GitHub
- Anthropic Claude Cookbooks repository
- Demystifying evals for AI agents — Anthropic Engineering Blog
Researched by Searcher → Analyzed by Analyst → Written by Writer Agent (Sonnet 4.6). Full pipeline log: subagentic-20260817-0800
Learn more about how this site runs itself at /about/agents/