AI Agent Token Budget Management: How Claude Code Prevents Runaway API Costs
Claude Code enforces hard token limits, compaction thresholds, and pre-execution budget checks. Here's how to implement the same pattern in your own agents.
The Real Cost Problem With AI Agents in Production
Every developer building with Claude or another frontier model eventually runs into the same moment: an agent that worked perfectly in testing suddenly runs up a $200 bill overnight. Token costs in multi-agent systems don’t scale linearly — they compound. Each tool call adds context. Each sub-agent response feeds back into the orchestrator. Without deliberate token budget management, a single runaway job can hit the context ceiling and start failing, or worse, keep retrying and burning through your API budget silently.
Claude Code, Anthropic’s terminal-based coding agent, takes a notably structured approach to this problem. It enforces hard token limits, automatically compacts conversation history before the context window fills, and runs pre-execution budget checks before kicking off expensive operations. These aren’t just nice-to-haves — they’re what separates a production-ready agent from a prototype that works until it doesn’t.
This article breaks down exactly how Claude Code’s token budget management works, what the design decisions reveal about building reliable agents, and how you can implement the same patterns in your own systems — whether you’re using the Claude API directly, LangChain, CrewAI, or a no-code platform like MindStudio.
Why Token Budget Management Is Harder Than It Looks
The naive approach to context management is to just let requests fail when they hit the limit, then handle the error. In practice, this creates a cascade of problems.
First, context window limits aren’t soft suggestions. Once a request exceeds a model’s maximum context length, the API returns an error. Your agent either crashes or, if it has retry logic, tries again — often with the same oversized context, burning tokens on failed calls.
Second, long contexts are expensive even when they succeed. GPT-4o and Claude Sonnet charge per input token on every request, not just for new tokens. A conversation that’s accumulated 100K tokens of history costs 100K input tokens on every subsequent call, regardless of how much new content you’re adding.
Third, long contexts degrade model performance. Research from Anthropic and others has consistently shown that models lose track of information in very long contexts — the “lost in the middle” problem. Keeping contexts artificially long doesn’t just cost more; it often produces worse outputs.
The Multi-Agent Amplification Effect
In single-agent systems, token accumulation is predictable. In multi-agent systems, it’s exponential. When Claude Code spawns sub-agents to handle file analysis, search, or code review, each sub-agent response gets fed back to the orchestrator. The orchestrator’s context grows with every round trip.
A three-agent pipeline where each agent produces 2,000 tokens of output can balloon the orchestrator’s context by 6,000 tokens per cycle. Over ten cycles, that’s 60,000 tokens of accumulated sub-agent output, plus the original task context, plus intermediate reasoning. This is exactly the scenario where token budget management stops being optional.
How Claude Code Approaches Token Budget Management
Claude Code handles token costs through three distinct mechanisms working in concert: hard context limits, automatic compaction, and pre-execution budget awareness. Understanding each one is useful for building similar logic into your own agents.
Hard Context Limits
Claude Code sets strict upper bounds on context size during initialization. Rather than running until the API returns an error, it tracks accumulated token usage actively and enforces its own ceiling before hitting Anthropic’s.
This distinction matters. API error handling is reactive — something already went wrong. Internal limit enforcement is proactive — you prevent the problem before it occurs. Claude Code uses the token count data returned in API responses to maintain a running tally of context size, then halts or compacts before that tally crosses a configurable threshold.
The threshold isn’t set at the model’s hard limit. It’s set lower — typically leaving a meaningful buffer — so there’s always room to generate a coherent completion or compaction summary before running out of space.
Automatic Context Compaction
When Claude Code’s context approaches its internal threshold, it triggers a compaction step. Compaction works by asking the model to generate a concise summary of the conversation so far — capturing key decisions, outputs, and state — then replacing the full history with that summary.
This is sometimes called “context summarization” or “memory compression” in other frameworks. The implementation details matter:
- The compaction prompt is crafted carefully to preserve task-relevant information while discarding conversational filler.
- The resulting summary is significantly shorter than the original context — typically by 60–80%.
- After compaction, the agent continues with the summary as its new context baseline, plus any immediately relevant recent messages.
Claude Code also exposes a manual compact command, letting users trigger compaction on demand before starting a large task. This is useful when you know an upcoming operation will be expensive and you want to start with a clean context.
The automatic trigger threshold is configurable. For smaller projects, you might set it to compact at 50% of the model’s context window. For tasks where preserving detailed history is important, you’d push that threshold higher and accept the associated cost.
Pre-Execution Budget Checks
Before kicking off any expensive operation — reading a large file, running a search across a codebase, spawning sub-agents — Claude Code evaluates whether the current context budget can support the expected operation.
This is the most sophisticated piece of the system. It requires the agent to estimate token costs before committing to an action. Claude Code does this through a combination of:
- File size heuristics: Estimating token count from byte count (roughly 1 token per 4 characters for English text, varies for code).
- Operation type awareness: Knowing that certain tools tend to produce large outputs and accounting for that.
- Remaining budget calculation: Subtracting current context size from the threshold to determine available headroom.
If an operation would exceed the remaining budget, Claude Code either asks for user guidance, skips the operation and notes the limitation, or triggers compaction first to free up space before proceeding.
This prevents the common failure mode where an agent partially completes a task, fills its context, and then crashes mid-operation — leaving state inconsistent and requiring manual cleanup.
The Token Budget Warning System
Claude Code surfaces token budget status to users throughout a session, not just when things go wrong. This is worth implementing in any production agent.
Progressive Warning Thresholds
Rather than a single alert when the context is nearly full, Claude Code uses tiered warnings:
- First warning at around 70% of the threshold — lets the user know compaction may happen soon.
- Second warning at 85% — signals that compaction is imminent.
- Action at 90% — triggers automatic compaction or halts with a clear message.
This gives users enough time to save state, finish a thought, or manually compact before the system has to intervene. It’s a better user experience than a sudden failure, and it’s a better engineering pattern than hoping the model figures it out on its own.
Surfacing Cost Alongside Capability
The other useful pattern in Claude Code is that token usage is surfaced as part of the agent’s self-awareness. The agent can be asked how much of its context budget remains, and it can factor that into its planning.
For example: “I have enough context budget to analyze three more files before compaction. Should I prioritize the main application logic or the test suite?” This turns budget constraints from invisible system failures into explicit planning parameters.
Implementing These Patterns in Your Own Agents
The mechanisms Claude Code uses aren’t proprietary. They’re design patterns you can implement with any LLM API. Here’s how to build each one.
Step 1 — Track Token Usage Actively
Most LLM APIs return token counts in their response objects. The Claude API returns usage.input_tokens and usage.output_tokens in every response. Use these to maintain a running total.
context_tokens = 0
threshold = 150_000 # Set below model's hard limit
def call_claude(messages, system=""):
global context_tokens
response = anthropic_client.messages.create(
model="claude-opus-4-5",
max_tokens=8096,
system=system,
messages=messages
)
context_tokens = response.usage.input_tokens
return responseDon’t estimate from message length alone — let the API tell you what it actually counted. Estimation is useful for pre-flight checks, but always update your tracked count from real API responses.
Step 2 — Build a Compaction Function
The compaction prompt is the core of this system. It should instruct the model to summarize the conversation while preserving key decisions, outputs, and task state.
def compact_context(messages, system_prompt):
compaction_prompt = """
Summarize this conversation concisely. Preserve:
- The original task and any clarifications
- Key decisions made and their rationale
- Files modified and their current state
- Any errors encountered and how they were resolved
- Current task status and remaining steps
Format as a structured summary an AI agent can use to continue the task.
"""
summary_response = call_claude(
messages=messages + [{"role": "user", "content": compaction_prompt}],
system=system_prompt
)
summary = summary_response.content[0].text
# Replace full history with summary
return [{"role": "user", "content": f"[CONTEXT SUMMARY]\n{summary}\n[END SUMMARY]"}]After compaction, reset your context_tokens counter based on the new context’s actual token count.
Step 3 — Add Pre-Execution Budget Checks
Before running any operation that might produce large outputs, estimate the token cost and compare it to remaining budget.
def estimate_tokens(text):
# Rough estimate: ~4 characters per token for English/code
return len(text) // 4
def check_budget(estimated_tokens, remaining_budget, operation_name):
if estimated_tokens > remaining_budget * 0.8:
return {
"proceed": False,
"reason": f"{operation_name} estimated at {estimated_tokens} tokens, "
f"but only {remaining_budget} tokens remain. Compact first."
}
return {"proceed": True}For file operations, use the file’s byte size to estimate tokens before reading. For search operations, use result count limits to cap output size. For sub-agent calls, use historical averages from past runs to set expectations.
Step 4 — Implement Tiered Warnings
Build budget status into your agent’s output loop rather than treating it as a background concern.
def get_budget_status(context_tokens, threshold):
pct = context_tokens / threshold
remaining = threshold - context_tokens
if pct >= 0.90:
return "critical", remaining
elif pct >= 0.85:
return "warning", remaining
elif pct >= 0.70:
return "notice", remaining
else:
return "ok", remaining
def agent_loop(messages, system):
while True:
status, remaining = get_budget_status(context_tokens, threshold)
if status == "critical":
messages = compact_context(messages, system)
continue
if status in ["warning", "notice"]:
print(f"[Budget {status.upper()}] {remaining} tokens remaining")
# Continue agent execution...Step 5 — Persist State Across Compactions
The biggest risk with context compaction is losing state that the agent needs to continue correctly. Before compacting, serialize any critical state to an external store.
For simple cases, this can be a local file. For production agents, it should be a database or key-value store that survives process restarts.
def save_agent_state(task_id, current_files, completed_steps, pending_steps):
state = {
"task_id": task_id,
"files_modified": current_files,
"completed_steps": completed_steps,
"pending_steps": pending_steps,
"timestamp": datetime.now().isoformat()
}
# Persist to your state store
state_store.set(f"agent_state:{task_id}", json.dumps(state))Pass the relevant portions of this state into the compaction summary so the model can reconstruct context accurately.
Where These Patterns Break Down
Token budget management solves real problems, but it introduces tradeoffs worth understanding before you implement it.
Compaction Loses Information
A summary is always a lossy compression. Details that seemed unimportant at compaction time might turn out to matter later in the task. Long-running agents that compact multiple times can drift from their original task requirements or forget edge cases the user specified early in the session.
The mitigation is to always include the original task specification in a persistent system prompt that survives compaction, and to structure your compaction prompt to explicitly preserve constraints, requirements, and anti-goals — not just positive outputs.
Threshold Tuning Is Task-Specific
A threshold that works well for a code review agent (where outputs are predictable in size) might be wrong for a research agent (where a single web page might dump 50K tokens of content into the context). You’ll need to tune thresholds per agent type, and possibly per operation type within the same agent.
Start conservative — compact early and often — and increase the threshold only when you find compaction is interrupting task flow unnecessarily.
Budget Checks Require Accurate Estimation
Pre-execution budget checks are only useful if your token estimates are roughly accurate. For structured operations (API calls with known response schemas), estimation is easy. For open-ended operations (web search, document parsing), it’s harder. Build in safety margins, and track estimation accuracy over time to improve your heuristics.
How MindStudio Handles Multi-Agent Token Management
Building this infrastructure from scratch is significant engineering work. For teams using MindStudio to build and run AI agents, a lot of this complexity is handled at the platform level.
MindStudio’s multi-agent workflows enforce execution boundaries between agents, which naturally prevents context accumulation from cascading across the whole pipeline. Each agent in a workflow runs with its own context scope. Data passed between agents is structured (typed inputs and outputs), not raw conversation history, so you’re not dragging entire context windows through the system with every handoff.
For developers building custom agents that need to call into MindStudio capabilities, the Agent Skills Plugin (@mindstudio-ai/agent) is worth knowing about. It exposes methods like agent.runWorkflow(), agent.searchGoogle(), and agent.generateImage() as typed method calls — meaning your Claude Code or LangChain agent can offload heavy operations to MindStudio without accumulating their outputs in the agent’s own context. The workflow runs externally, returns a structured result, and your agent’s context stays lean.
This is particularly useful for the class of problems where a single tool call might generate a large intermediate output — web scraping, document parsing, media generation — that you don’t actually need in full inside your agent’s reasoning context.
You can start building with MindStudio free at mindstudio.ai.
Frequently Asked Questions
What is context compaction in Claude Code?
Context compaction is the process of replacing a long conversation history with a concise summary. When Claude Code detects that accumulated context is approaching its configured limit, it generates a structured summary of the session — preserving key decisions, outputs, and task state — then discards the full history. The agent continues from the summary, which is significantly shorter. This prevents the context window from filling completely and allows long-running tasks to continue without failure.
How do I set token limits for Claude API requests?
You control token usage at two levels. The max_tokens parameter in each API request sets the maximum number of tokens the model can generate in a single response. To manage total context size across a multi-turn conversation, you need to track cumulative input tokens yourself using the usage field in API responses, then implement compaction or history truncation when you approach your threshold. Claude’s maximum context window varies by model — Claude Sonnet 4 and Claude Opus 4 support up to 200K tokens.
Why does my AI agent cost more in production than in testing?
The most common reason is context accumulation. In testing, each run starts with a fresh context. In production, agents often run in long sessions or across many iterations, accumulating history with each step. The cost of each API call scales with total input tokens, so a context that’s grown to 80K tokens costs 80K input tokens on every subsequent request — even if you’re only adding a few hundred new tokens per turn. Implementing context compaction and session boundaries directly reduces this cost.
What’s the difference between token limits and rate limits?
Token limits refer to the maximum number of tokens in a single request (the context window limit) or the maximum tokens a model can generate in one response. Rate limits are restrictions on how many requests or tokens you can send per minute or per day, set by the API provider. Token budget management in the Claude Code sense primarily addresses context window limits — keeping individual requests within bounds. Rate limits are a separate concern managed through request throttling and backoff logic.
How does Claude Code decide when to compact automatically?
Claude Code triggers automatic compaction based on a percentage threshold of the model’s context window, not a fixed token number. When tracked input tokens approach this threshold (typically around 85–90% of the configured limit), the compaction routine runs automatically. The exact threshold is configurable. Users can also trigger compaction manually using the /compact command before starting a large task, which is useful for clearing accumulated context from exploratory work before beginning a focused task.
Can compaction cause an agent to lose important information?
Yes, and this is the main tradeoff of context compaction. A summary is a lossy compression — details that seem peripheral at compaction time might be needed later. To minimize this risk, always keep the original task specification in a persistent system prompt that’s not subject to compaction, structure your compaction prompts to explicitly preserve constraints and requirements, and persist critical state to external storage (files, databases) before compacting. For tasks where full history is essential — like legal review or audit trails — reconsider whether compaction is appropriate and look at alternative strategies like chunked processing instead.
Key Takeaways
- Token costs in multi-agent systems compound quickly — each sub-agent response feeds back into the orchestrator’s context, and input tokens are charged on every request regardless of how much new content you’re adding.
- Claude Code’s token budget system works through three mechanisms in concert: hard internal limits (set below the API’s ceiling), automatic context compaction at configurable thresholds, and pre-execution budget checks before expensive operations.
- Context compaction — replacing full conversation history with a structured summary — can reduce context size by 60–80%, allowing long-running agents to continue without failure.
- The same patterns are implementable with any LLM API: track token usage from API responses, build a compaction function with a carefully crafted prompt, add pre-flight budget checks for large operations, and persist critical state to external storage before compacting.
- For teams building on managed platforms, structured agent handoffs (typed inputs/outputs between agents rather than raw context passthrough) provide a natural form of context isolation that prevents accumulation across the pipeline.
Managing token budgets is one of those things that feels optional until it isn’t. Getting the architecture right early means your agents stay reliable and predictable as usage scales — and your API bill stays within reason.
