How Claude Opus 5 Cracked ARC-AGI-3 With Algebraic Reasoning
Opus 5 jumped from single digits to 30% on ARC-AGI-3 by converting visual puzzles into algebra, a new reasoning behavior researchers hadn't seen before.
What is ARC-AGI-3 actually testing?
ARC-AGI-3 is the latest version of a benchmark created by AI researcher Francois Chollet, designed to measure something most standard benchmarks ignore: how well a system handles situations it has never seen before. Instead of quizzing a model on facts or patterns it likely absorbed during training, ARC-AGI-3 drops it into unfamiliar, game-like environments with no instructions. No rules, no tutorial, no labeled goal. The system has to figure out what it’s controlling, what the objective is, and how to get there, purely through interaction.
This distinction maps onto a longstanding idea in cognitive science: crystallized intelligence versus fluid intelligence. Crystallized intelligence is what you’ve learned and can recall. Fluid intelligence is your capacity to reason through something brand new. Large language models have always been strong on the crystallized side, since they’re trained on enormous amounts of existing knowledge. Fluid intelligence, adapting on the fly to a genuinely novel task, has been the harder problem. ARC-AGI-3 exists specifically to measure that gap, under the working slogan “easy for humans, hard for AI.”
TL;DR
- Claude Opus 5 scored 30.2% on ARC-AGI-3, far ahead of the prior best of 7.8% from GPT-5.6-based models and well above Anthropic’s own Opus/Sonnet family, which had been sitting near 20%.
- Arc Prize researchers reported a novel behavior they hadn’t observed in any frontier model before: Opus 5 converted visual puzzle layouts into explicit algebraic notation before solving them.
- The model didn’t just solve puzzles, it solved five previously unbeaten environments, matching or beating human-level sample efficiency on four of them, meaning it needed roughly as few attempts as a person would.
- ARC-AGI-3 tasks are presented to models as structured data (JSON-like text), not images, so the “algebraic notation” behavior is a reasoning strategy applied to symbolic descriptions, not a vision trick.
- The jump has triggered real skepticism in the AI community, with critics pointing out that turning a puzzle into algebra is a learnable, gameable tactic rather than proof of general intelligence.
- Opus 5 launched at roughly half the price of Anthropic’s larger Fable-class model while matching or beating it on several benchmarks, continuing a pattern where newer, cheaper models leapfrog bigger ones.
- The gain arrived only about two months after the previous Opus release, underscoring how fast capability jumps are happening at the frontier right now.
- ✕a coding agent
- ✕no-code
- ✕vibe coding
- ✕a faster Cursor
The one that tells the coding agents what to build.
How did Opus 5 solve puzzles nothing else could?
Arc Prize’s own writeup describes the mechanism directly: Opus 5 used logical reasoning to turn ARC-AGI-3 layouts into algebraic notation. In one documented example, the model described a scene using an equation-like structure, something along the lines of expressing one object’s position as a function of another’s, effectively writing “four center equals two times axis minus five center.” A few hundred actions later, in a different puzzle, it generalized this into a two-dimensional relationship, describing a duplicated or mirrored object as a “reflection about an axis.”
What’s happening here is that Opus 5 isn’t just pattern-matching pixels or guessing at rules through trial and error. It’s building a compact mathematical model of how objects in the environment relate to and move with each other. Once it has that model, in the form of an actual equation, it can predict outcomes and plan actions instead of blindly exploring. That’s the difference between fumbling around a new game for hundreds of tries and recognizing “oh, this piece mirrors that piece across this line” and then acting on it directly.
This matters for a specific technical reason: sample efficiency. Humans encountering a new game don’t try a million random inputs. We form a working theory of the rules quickly and refine it. Arc Prize measures this explicitly, and Opus 5 didn’t just clear five previously unsolved environments, it did so while matching or beating human-level efficiency on four of them. That’s the headline claim in the source reporting, and it’s a meaningfully higher bar than simply “getting the right answer eventually.”
Is this actually a vision breakthrough?
No, and this is a common misunderstanding worth correcting. ARC-AGI-3 environments are not fed to language models as images. The puzzles are presented in structured text or JSON format describing objects, positions, and states. So when people say Opus 5 “turned a visual puzzle into algebra,” the model was never doing visual perception in the first place. It was reasoning over symbolic, text-based representations of a scene and choosing to compress that representation into algebraic relationships rather than tracking every object’s position individually.
That’s still notable. Choosing to abstract a scene into a formula, rather than reasoning step by step about raw coordinates, is a strategic move that Arc Prize says it hadn’t seen from a frontier model before. It suggests something about how Opus 5 approaches unfamiliar structured problems generally, not just this specific benchmark.
Why are people skeptical of the ARC-AGI-3 jump?
The scale of the improvement, from roughly 7.8% (the prior best, associated with a GPT-5.6 variant) up to 30.2%, is large enough that it has drawn immediate scrutiny rather than uncritical celebration. Several points of pushback have circulated:
Some in the AI community argue that turning a visual or structured puzzle into explicit algebra is a learnable, somewhat mechanical tactic once you know it works, not evidence of deep general reasoning. There are also reports that a similar strategy, achieved through scaffolding and prompting techniques rather than model retraining, had already been used to push an older Claude model (Opus 4.6) toward better ARC-AGI-3 performance. If that’s accurate, it implies the underlying trick isn’t unique to whatever changed inside Opus 5, and could partly reflect better training exposure to this kind of problem-solving pattern.
There’s also the broader “benchmaxing” concern that shows up every time a model posts an outlier score on one popular benchmark while looking merely competitive, or even behind, on others. Other reported benchmark comparisons show Opus 5 trailing larger models in certain domains even as it leads decisively on ARC-AGI-3. None of this necessarily invalidates the result. Arc Prize’s team has historically been conservative and reluctant to declare victories prematurely, and they explicitly flagged this as a new behavior worth watching rather than a solved benchmark. But it’s a reasonable caution against reading a single 4x jump as proof that fluid, general problem-solving has been cracked.
Does this change what “reasoning” means for AI models?
It’s a data point rather than a verdict. What ARC-AGI-3 is designed to catch is whether a system can walk into a completely unfamiliar structured environment, with zero instructions, and figure out the rules efficiently. Opus 5 clearing previously unbeaten environments at close to human sample efficiency is a genuinely different kind of result than incremental gains on knowledge-heavy benchmarks like coding or math tests, where more training data and bigger models reliably help.
The algebraic notation behavior specifically suggests that at least in some cases, a large model given a novel symbolic environment will choose to build an internal formal model of it rather than rely purely on trial and error or memorized patterns. Whether that generalizes broadly, to messier real-world tasks with more noise and ambiguity, is exactly the open question the skeptics are raising. For now it stands as one of the more concrete, well-documented examples of a frontier model doing something researchers hadn’t specifically trained for or previously observed.
Frequently Asked Questions
What is ARC-AGI-3?
It’s a benchmark created by researcher Francois Chollet’s Arc Prize organization that drops AI systems into novel, instruction-free interactive environments to measure how efficiently they can learn new rules and achieve goals, as opposed to recalling patterns from training data.
How much did Opus 5 improve on ARC-AGI-3?
Opus 5 scored 30.2%, up from a previous best of 7.8% held by a GPT-5.6 based model, and above Anthropic’s own larger Fable-class models, which scored around 20%.
What does “turning puzzles into algebra” mean?
Opus 5 represented relationships between objects in a puzzle (positions, mirrored movements, and similar structures) as explicit equations rather than tracking every object individually, then used that compressed mathematical model to plan its actions.
Is ARC-AGI-3 a vision test?
No. Puzzle environments are given to models as structured text or JSON data describing objects and states, not as images, so this is a reasoning behavior over symbolic input rather than a visual perception capability.
Why do some people distrust the ARC-AGI-3 score?
Critics point out that converting a puzzle into algebra is a learnable tactic that reportedly worked with scaffolding on an older Claude model too, and note that Opus 5 doesn’t lead on every benchmark, raising the possibility that this specific benchmark was disproportionately optimized for.