METR's Anthropic Audit: Is AI Really Speeding Up AI Research?
METR's independent audit found Anthropic's AI progress has nearly doubled in pace, with most of it credited to AI itself. Here's what that means.
What did METR actually find inside Anthropic?
METR, a nonprofit known for benchmarking how long AI agents can work autonomously, spent from August 12 to October 29, 2025 embedded inside Anthropic with access to internal benchmark scores, training data, compute figures, staff surveys, and interviews with about 15 researchers. The resulting report, published November 5 and titled “Independent Assessment of AI R&D Acceleration at Anthropic,” concluded that Anthropic’s model capabilities are improving at roughly twice the rate they were between March 2024 and October 2025, that AI itself likely accounts for around 80% of that speed up, and that METR found no clear bottleneck likely to slow this down within the next year.
TL;DR
- METR’s headline number is a 1.9x acceleration in Anthropic’s capability gains since October 2025, compared to the prior rate from March 2024 to October 2025.
- The uncertainty range runs from 1.5x to 2.8x, and METR said it wouldn’t be shocked if the true figure were as low as 1.3x or as high as 3x.
- METR attributes roughly 80% of the speed up to AI itself helping with research, not to a jump in training compute, which it explicitly ruled out as the driver.
- The report references unreleased internal checkpoints called model two, model three, and model four, with model four finishing training between September and October 2025 and scoring above 50% on all 15 benchmarks METR used.
- Anthropic staff surveys describe self-reported productivity gains climbing from around 1.2x with Claude 3.7 Sonnet to roughly 12x with model four, though METR flags this as weak, self-reported evidence.
- Anthropic redacted large sections of the report, including the evidence METR found most informative about bottlenecks and the compute analysis, so outside readers see the conclusions but not most of the supporting detail.
- Google DeepMind CEO Demis Hassabis has separately described this exact pattern, AI making human researchers several times more productive while humans stay in the loop, as the early, lower-risk phase of recursive self-improvement.
- Patient records✓ Granted
- Surgery schedule✓ Granted
- Billing export✕ Denied
What is the AECI benchmark and why does the slope matter?
AECI stands for Anthropic’s version of the Epoch Capabilities Index, a composite score built from multiple benchmarks so researchers can track capability gains across models over time on a single chart. METR’s central chart plots AECI scores against the dates when most Anthropic employees gained internal access to each model, not public release dates. Before October 2025, Anthropic’s models gained about 15 AECI points per year. After October 2025, that climbed to about 29 points per year, nearly double. METR’s own framing is blunt: Anthropic appears to have packed roughly as much benchmark progress into the past year as it did across the prior two years, a span that covered the jump from Claude 2.1 to Sonnet 4.5.
It’s worth being precise about what “2x acceleration” means here. It doesn’t mean models got twice as capable. It means the rate of improvement doubled, so whatever amount of progress used to take two years now takes about one. Compounding is the part that worries researchers: if the growth rate itself keeps climbing, the gap between what’s possible and what’s released could widen fast.
Is this driven by more compute, or by something else?
METR says it’s not compute. The report states it received “convincing evidence” that the acceleration was not due to faster growth in training compute, though the specific evidence behind that conclusion was redacted. That distinction matters because it shifts the explanation away from the traditional story of AI progress (bigger clusters, more chips, bigger training runs) toward a different one: AI systems helping researchers work faster.
METR’s rough estimate is that about 80% of the current acceleration comes from AI assisting with research itself, a number the report says carries real uncertainty but has reportedly been rising over time. If that holds, it means the primary lever on AI progress is shifting from capital spending to the models’ own usefulness as research tools, a loop where better models make it faster to build the next better model.
What are Anthropic’s unreleased “model two, three, and four”?
METR’s report references internal checkpoints the public has never used: model two (finished training July 2026, per the transcript’s timeline), model three (August to September), and model four (September to October). Model four is described as the highest-scoring model in METR’s full set, clearing 50% on all 15 benchmarks tested and reportedly pushing past 90% on several. Commentators tracking the report on social media noted the gap between Anthropic’s internal frontier and its public releases has reportedly stretched past 40 days at points, meaning the most capable model inside the company isn’t necessarily the one available to paying users. Some observers speculated model four could become a future Claude release, but that’s speculation, not something METR’s report confirms.
How much faster does AI make Anthropic’s researchers?
METR gathered informal Slack polls and interviews asking staff how much AI was speeding up their own work. Self-reported multipliers climbed sharply over time: around 1.2x in early 2025 with Claude 3.7 Sonnet, above 5x by the time of a model internally referred to as Mythos 5, and around 12x by October 2025 with model four. METR is careful to label this soft evidence, since it’s self-reported and people are bad at estimating their own productivity. Interview quotes varied widely: one researcher said they were “only bottlenecked on ideas and compute, no longer bottlenecked on code,” while another pushed back hard, estimating only 15 to 20% of their speed up came from AI and saying they don’t trust self-reported productivity claims at all. METR notes the more structured interview answers tended to cluster closer to a 2x speed up, a more modest but still notable figure.
Crucially, METR found AI is still mostly augmenting human researchers rather than running research autonomously, though it noted early signs of more independent AI contributions emerging.
Does this mean recursive self-improvement has started?
That’s the debate the report triggered. Recursive self-improvement, the idea that AI helping build better AI creates a compounding loop, has been a long-standing concern among AI safety researchers because a fast enough loop could outpace anyone’s ability to check whether the resulting systems are safe. METR’s findings (rising acceleration, most of it attributed to AI, no clear 12-month bottleneck) fit the early description of that loop, but they don’t describe a closed system running without humans.
That distinction lines up with how Demis Hassabis has framed it publicly: an early, human-in-the-loop phase where AI makes engineers several times more productive but people still review and architect the work, versus a later and riskier “closed loop” phase where the review step itself gets automated away. By that framing, what METR documented inside Anthropic looks like the earlier, lower-risk stage. The open question is how quickly that changes, especially since Anthropic CEO Dario Amodei said earlier in 2025 that AI coding tools were giving the company something like a 15 to 20% overall speed up, up from roughly 5% six months prior, a far smaller number than the roughly 2x figure METR later reported.
Why did Anthropic redact parts of the report?
METR’s public report is a condensed version of a roughly 100-page internal document, and several sections, including detailed evidence on compute growth and on potential bottlenecks, were redacted before publication. This means outside readers can see METR’s conclusions (2x acceleration, 80% AI-attributed, no near-term bottleneck) without seeing the underlying data that produced them. That’s part of why the report has generated so much discussion: an independent group got unusually deep access inside a frontier lab, but the public is still relying on that group’s summary rather than the raw evidence.
Frequently Asked Questions
What is METR and why does its assessment carry weight?
METR is a nonprofit AI research organization known for benchmarking how long autonomous AI agents can complete tasks without human help, a metric widely cited across the industry. Its access to internal Anthropic data (benchmarks, compute figures, staff interviews) for this report was unusually deep, which is why its conclusions drew significant attention.
What does “AI R&D acceleration” actually mean?
It refers to the rate at which AI capabilities are improving over time, not the capabilities themselves. A 2x acceleration means progress that previously took two years is now happening in about one, largely because AI tools are reportedly making the humans building AI faster.
Is Anthropic’s “model four” the same as a future Claude release?
METR’s report doesn’t say that. It only describes model four as an internal checkpoint that finished training between September and October 2025 and scored highest among the models METR analyzed. Claims that it will become a specific named public release are speculation from outside commentators, not something confirmed in the report.
Did more computing power cause this acceleration?
METR says no. It reported receiving evidence that the acceleration was not driven by increased training compute growth, though the specific evidence was redacted. The report instead attributes most of the speed up to AI assisting research work directly.
Does this report prove recursive self-improvement is happening?
It shows a pattern consistent with an early stage of it, AI significantly speeding up the humans who build AI, with no clear near-term bottleneck in sight. It does not show AI improving itself without human involvement, which researchers like Demis Hassabis describe as a distinct and riskier later stage.
