Inside OpenAI's Million-Line Codebase Built Almost Entirely by AI Agents
Three OpenAI engineers used Codex agents to ship a million-line, 1,500-PR internal product. Here's how they structured the work.
What did OpenAI actually build with AI agents?
Three engineers at OpenAI shipped an internal product using Codex agents instead of writing the code by hand. The project grew to more than a million lines of code and involved roughly 1,500 pull requests, completed in a fraction of the time a traditional hand-coded build would have taken. Individual Codex sessions ran for six hours or longer at a stretch, with agents handling implementation while the humans focused on direction and review.
The headline number (a tenth of the usual build time) is eye-catching, but the more useful story is how the team kept a fleet of long-running agents pointed at a moving target without the project collapsing into noise. That’s the part worth studying if you’re trying to use agents for anything bigger than a single coding task.
TL;DR
- OpenAI engineers used Codex agents to build an internal product that reached over a million lines of code across about 1,500 pull requests, without engineers typing the implementation themselves.
- The team hit a common problem with long agent runs: a single giant instruction file eventually turns into a graveyard of stale rules that crowds out the actual task at hand.
- Instead of one master document, they used a short map that pointed agents to active execution plans, decision logs, design documents, and an architecture map that stayed current as the project evolved.
- The core technique, sometimes called progressive context shaping, means updating the small set of instructions governing “what happens next” as the work reveals new information, rather than defending the original prompt.
- Anthropic has converged on a similar pattern for long-running Claude sessions, using a progress file that records completed work, current state, and failed approaches so new sessions don’t repeat dead ends.
- Separating stable instructions, current state, a map of resources, and history keeps agents from either forgetting critical context or drowning in irrelevant past events.
- The technique generalizes across tools: it works in Codex, Claude Code, and even ordinary chat sessions, because the fix is about how context is structured, not which model runs it.
Remy is new. The platform isn't.
Remy is the latest expression of years of platform work. Not a hastily wrapped LLM.
Why does a giant instruction file break down on long agent projects?
Long agent runs accumulate information fast. Over six or eight or ten hours, an agent produces code, test results, dead ends, partial decisions, and new facts about the problem it’s solving. The natural instinct is to keep adding rules to a single master file so the agent “remembers” everything.
That approach fails for a structural reason: every instruction in that file competes for the agent’s attention on every turn, whether or not it’s still relevant. Early guidance that made sense before the work started can become actively wrong once the project reveals new constraints, but it doesn’t disappear from the file. It just sits there, competing with the current task for the agent’s limited attention. OpenAI’s team described the result plainly: a graveyard of stale rules.
The fix isn’t a bigger context window. It’s separating what needs to stay fixed (guardrails, approval requirements, coding standards) from what needs to update constantly (the current goal, active decisions, what to do next). OpenAI replaced its long manual with a short map that pointed Codex toward the documents that mattered right now: execution plans, decision logs, design documents, an architecture map, and quality grades for different parts of the codebase. As the product changed, those documents changed with it, so the agent was always reading the current version of the plan instead of the original one.
How do you steer an agent mid-project without restarting it?
The method here isn’t about writing a better initial prompt, though that still matters. A strong opening prompt should define the desired result, name the materials the agent can use, set clear boundaries on what it can and can’t do, and specify what requires human approval before it happens (publishing something, spending money, deleting data, contacting another person).
But an opening prompt can only encode what you know before the work starts. Once an agent has run for hours and produced real output, the job shifts. You’re not defending the original brief anymore, you’re updating what the agent treats as the current version of the assignment. That’s the practical difference between placing an order and directing ongoing work.
This shows up in a few concrete forms:
A current state file (often a markdown file, sometimes a ticket or JSON record) holds the live version of the goal, which decisions are locked in, what’s unresolved, and when the agent should stop. Claude Code can be told to read a file like this at the start of a session and update it after major decisions. Codex can use an agents.md file the same way. In an ordinary chat, the equivalent move is asking for a fresh project brief that reflects what’s changed, then carrying that brief into the next conversation instead of the old thread.
Other agents start typing. Remy starts asking.
Scoping, trade-offs, edge cases — the real work. Before a line of code.
The test for whether something belongs in that file is simple: if a fresh agent picking up the work later would miss a decision you care about, it goes in the current state. If it only explains how you got there, it belongs in history instead, not in the active instructions.
What does Anthropic’s approach to long Claude sessions have in common with this?
Anthropic has arrived at a similar structure independently, for a different reason: long-running scientific computing tasks in Claude Code. Their agents use a progress file as portable memory between sessions. It records what’s been completed, what the current state is, and which approaches failed and why. A new Claude session can read that file, understand that a particular method turned out “too stiff” (to use one documented example) and move straight to the replacement approach, instead of re-running the failed attempt from scratch.
The shared insight across OpenAI’s Codex project and Anthropic’s Claude work is that AI use is shifting from single-turn answers to work that spans hours or days across many separate sessions. Once that’s true, context can’t be a static packet you write once and never touch. It has to evolve as the work teaches you what the actual problem is.
What four types of context should you keep separate?
A useful way to organize this, drawn from how OpenAI and Anthropic structured their long-running projects, is to split context into four categories:
Stable instructions cover how to work: where to find things, what requires approval, what quality standards apply. This changes rarely and can live in a file like agents.md or claude.md.
Current project state covers what the goal means right now: active decisions, open questions, what should happen next, and when to stop. This changes constantly and should carry more authority than anything else the agent reads.
The map tells the agent what resources exist and where, without dumping all of it into the context window at once. Research files, design docs, prior drafts. The agent needs a reliable way to locate the right piece for the next decision, not the whole library at once.
History covers what happened and why: git logs, decision logs, changed approaches. It matters for auditing and recovery, but it shouldn’t compete with current instructions for the agent’s attention.
Skipping the “map” layer is a common failure mode. Without it, people default to pasting everything into the prompt, which recreates the exact stale-rules problem this structure is meant to avoid.
Is this approach worth adopting outside of huge engineering projects?
Yes, and the scale doesn’t have to be a million lines of code to benefit. The AI research group Arise documented a smaller but telling example: an agent tasked with summarizing multiple traces made 27 model calls, most of which went into reorganizing its own to-do list rather than finishing the task. The original request had been buried under its own intermediate output. Arise’s fix wasn’t a stronger instruction at the top of the prompt. It was moving the current plan outside the conversation entirely, storing it on disk, and rebuilding a short plan message from the latest state before every model call. The noisy transcript stayed available, but the maintained plan got priority.
Everyone else built a construction worker.
We built the contractor.
One file at a time.
UI, API, database, deploy.
That’s the same principle at a much smaller scale: separate the record of what happened from the live instruction of what to do next, and give the second one more weight.
Frequently Asked Questions
How many engineers worked on OpenAI’s million-line agent project?
Three engineers led the project, using Codex agents to handle the implementation work rather than writing the code themselves.
How long did individual Codex sessions run in this project?
Individual runs stretched past six hours at a stretch, which is long enough that context management became a central engineering problem rather than a minor detail.
What is “progressive context shaping”?
It’s the practice of starting an agent project with a clear brief, then continuously updating the small set of active instructions and decisions as the work reveals new information, rather than treating the original prompt as fixed.
Does this only work with Codex and Claude?
No. The pattern (separating stable rules, current state, a resource map, and history) applies to any long-running agent workflow, including ordinary chat sessions where you periodically rewrite the project brief instead of relying on the full thread history.
What’s the risk of not managing agent context this way?
A long-running agent can carry a wrong direction just as far as a right one. Without an updated current state, an agent can spend hours producing output based on an assumption that stopped being true early in the run.