Billion Context: Setting Up Infinite-Length Claude Code Sessions
How to install the Billion Context proxy and wire it into Cline Pass and Claude Desktop so long coding sessions stop overflowing or costing too much.
What is Billion Context and what problem does it solve?
Billion Context is an open source proxy that sits between your coding agent and the model API, compressing conversation history so long sessions stop hitting context window limits. Instead of truncating or blindly summarizing everything at once, it gives the model four tools (compress, decompress, search context, and status) so the model itself decides what to fold away and what to keep. The project has grown fast on GitHub, reportedly passing 3,700 stars and 300 forks within about three months, and it targets a specific pain point: long-running agent sessions that either blow past their context window or resend the same bloated history on every single turn, racking up token costs along the way.
TL;DR
- Billion Context is a proxy, not a plugin baked into one agent. You point your agent’s base URL at it, and every request gets compressed before it reaches the real model.
- The compression is model-driven and reversible: the agent gets compress, decompress, search context, and status tools, so it chooses what to summarize and can pull exact details back later.
- Reported benchmarks on DeepSeek V4.1 Flash show the compressed approach scoring slightly better than full context on a coding benchmark (F1 of 0.31 vs 0.29) while costing about 58% less.
- On a long-memory benchmark it scored lower than full context overall but well ahead of a retrieval-augmented setup, and in an extreme 1.48 million token test it reportedly survived every run while full context failed all 199 attempts.
- The tradeoff is exact recall: needle-in-a-haystack style retrieval of one precise sentence reportedly dropped sharply, since summaries preserve meaning, not verbatim text.
- It installs as a global npm package and can run as a native plugin, a client launcher, or a generic proxy you point any OpenAI or Anthropic-style base URL at.
- Pairing it with Cline Pass (a flat-rate plan for open coding models like DeepSeek V4.1 Flash) is a practical way to stretch session limits further, since compression reduces how much history gets resent on every turn.
How does the compression actually work?
Billion Context doesn’t cut off old messages or produce one giant blurry summary the way built-in agent compaction often does. It exposes four tools directly to the model:
- Compress folds a range of messages into a detailed summary.
- Decompress restores that range when the exact details are needed again.
- Search context runs a keyword search across everything that has been compressed.
- Status (referred to as ACP status) reports how full the context is and what’s still available to compress.
The model decides when and what to compress, rather than a hardcoded rule firing at some fixed token threshold. Compression also happens incrementally in small ranges instead of all at once, which keeps the prompt cache largely intact. According to the project’s own reporting, a healthy session can sit at a 95 to 97% cache hit rate, with the compression step itself adding 2% or less overhead. That matters because cached tokens are much cheaper than fresh ones, so preserving the cache is a big part of why the approach saves money.
What do the benchmarks actually show?
The repository includes a paper describing roughly four and a half months of real-world use: about 174,000 model calls and 18.76 billion input tokens, with zero context window overflows reported on 200k-token models.
A separate third-party benchmark ran on DeepSeek V4.1 Flash using Context Bench, a coding-agent benchmark built around exploring real repositories. Billion Context reportedly scored an F1 of 0.31 versus 0.29 for full context, at a cost of $4.61 versus $11.05, a roughly 58% cost reduction with a slightly better score.
On a long-memory benchmark (Long Mem AI), it scored 79.2% compared to 83.2% for full context, but both beat a retrieval-augmented generation (RAG) baseline at 62.6%, while running at about 71% lower cost than full context.
In a stress test using a 1.48 million token session, full context reportedly failed all 199 attempts, while the compressed approach survived every run, achieving roughly 30x compression.
The clear weak spot is precision retrieval. On a needle-in-a-haystack test, where the task is pulling back one exact sentence from deep in the history, accuracy reportedly dropped from 69% to 27%. Summarization keeps the gist, not the exact wording, so if a coding agent needs a precise line of code or config value from far back, it has to explicitly decompress that range first.
How do you install and set it up?
Billion Context installs as a global npm package:
npm install -g billion-context --prefix <your-local-folder>This gives you a command called billy. From there, three setup paths exist:
- Native plugin install: run
billy plugin installwith a client name for agents that support it directly, such as Pi, Open Code, Kimi Code, or Hermes. - Client launcher: run
billy claudeorbilly codexto launch that client through the proxy automatically, without editing any config files. - Manual proxy: start
billyon its own and point any OpenAI or Anthropic-compatible base URL at the local proxy address. This is the path needed for Cline and most other agents not on the native plugin list.
Once running, the proxy exposes a local dashboard (by default at localhost:8787) showing active sessions, live logs, and a config editor. Typing a status command like /acp cache in a supported client returns a report on cache hit rate, tokens saved by compression, and what might have broken the cache if the hit rate dropped. That report is the most direct way to confirm the proxy is actually saving money on a given session, rather than just assuming it is.
How does it fit with Cline Pass and Claude Desktop?
Cline Pass is a flat monthly plan that provides access to open coding models including DeepSeek V4.1 Flash, Kimi K3.1 Pro, GLM 5.3, Qwen 38 Max, and MiniMax M31 Flash, with higher rate limits than standard access. It also works outside the Cline app through an OpenAI-compatible API, using a key generated from the Cline dashboard.
To route Cline through Billion Context, you select an OpenAI-compatible provider, set the base URL to the local proxy address followed by the Cline API endpoint, paste in your Cline API key, and set the model to the Cline Pass version of DeepSeek V4.1 Flash. Every request then flows through the proxy for compression before hitting Cline Pass.
The same setup reportedly works in Claude Desktop, the standalone Claude app that supports parallel sessions, scheduled jobs, and importing conversations from Claude Code or Codex to continue them. Claude Desktop’s provider list includes an OpenAI-compatible option, so the same base URL, key, and model configuration applies there too.
The logic for pairing these together is straightforward: flat-rate plans like Cline Pass still enforce rate windows (hourly, weekly, monthly). A long session that resends its entire history on every turn burns through those limits fast. Cutting that resent history down with compression means the same plan budget stretches across more actual work.
Is Billion Context worth setting up right now?
It depends on how long-running your agent sessions typically are and how much verbatim recall you need. For coding agents working through large codebases over extended sessions, the benchmarks suggest a real cost reduction with comparable or slightly better task performance, and dramatically better reliability as sessions grow into the millions of tokens. For tasks that depend on pulling back one exact line of text from deep history, the measured drop in precise recall is a real limitation worth knowing about before relying on it for anything where exact wording matters.
There are also some practical caveats. The project moves fast (versioned past 0.12 and updating itself by default), so sticking to npm releases rather than the development branch is safer. Running it exposed on a network with a host flag enabled reportedly has no built-in authentication, so that should only be done on a trusted network. And at least one integration point, the code tab inside Claude Desktop’s own app, apparently bypasses the proxy entirely, meaning no compression happens there.
Frequently Asked Questions
What models does Billion Context work with?
It’s designed to work with any Anthropic or OpenAI-compatible API, since it acts as a proxy in front of the real model endpoint. The transcript highlights DeepSeek V4.1 Flash specifically because that’s the model used in the third-party benchmark, but the proxy itself isn’t tied to one model family.
Does Billion Context replace my agent’s built-in summarization?
Yes, in practice it’s meant to be a better alternative. Built-in compaction in most agents collapses the whole history into one summary at a fixed trigger point. Billion Context compresses incrementally in smaller ranges, keeps the result reversible through its decompress tool, and preserves more of the prompt cache, which built-in summarization typically disrupts.
Is there a cost to using the compression itself?
According to the project’s reporting, the compression step adds roughly 2% or less in overhead cost, which is small relative to the savings from avoiding repeated full-history token charges and from preserving cache hits.
What’s the biggest tradeoff with this approach?
Precision recall. Summarized history keeps the meaning but not exact wording, so retrieving one specific sentence or exact value from far back in a session is less reliable unless the model explicitly decompresses that section first.
Can I use this with Cline Desktop or other GUI orchestration tools?
Yes. The setup described routes both Cline and Claude Desktop through the same local proxy using an OpenAI-compatible base URL and a Cline Pass API key. Orchestration tools that use multiple underlying agents (for example, separate worker agents running on Pi, Open Code, or Kimi Code) can also route through Billion Context if those clients have native plugin support.
