What Is ARC AGI 3? The Interactive AI Benchmark Humans Solve at 100%
ARC AGI 3 is a video game-style benchmark where humans score 100% and every frontier AI model scores 0%. Here's how it works and why it matters.
The Benchmark That Stops Every AI in Its Tracks
There’s a benchmark where every frontier AI model — GPT-4o, Claude, Gemini, and the rest — scores exactly 0%. Meanwhile, average humans walk through the same tasks and score 100%. That benchmark is ARC AGI 3, and it’s designed to expose a fundamental gap that most AI evaluations quietly paper over.
This isn’t a trick question or an obscure edge case. ARC AGI 3 is a video game-style reasoning benchmark built specifically to test a type of intelligence that current AI systems don’t have: the ability to learn rules by interacting with an environment in real time.
This article explains what ARC AGI 3 is, how it works, what it measures, and why a 100% human / 0% AI split is one of the most significant results in AI evaluation today.
Background: The ARC Series and Why It Exists
ARC stands for Abstraction and Reasoning Corpus. The benchmark series was created by François Chollet — the researcher who built Keras and has spent years arguing that mainstream AI progress metrics often measure the wrong things.
The core concern
Chollet’s argument is that most AI benchmarks test memory and pattern matching, not genuine reasoning. A model that has trained on billions of text and image examples can recognize familiar patterns at superhuman speed. But that’s different from actually understanding a problem well enough to solve it from scratch, in a context the model has never encountered.
His 2019 paper, On the Measure of Intelligence, laid out a framework for measuring general intelligence as the ability to efficiently acquire new skills — especially in genuinely novel situations. ARC was built on that framework.
What makes ARC tasks different
ARC tasks are designed to be:
- Novel enough that memorization won’t help
- Solvable by any human without specialized knowledge
- Based on core reasoning priors every person has: object permanence, basic geometry, symmetry, counting, spatial relationships
The benchmark isn’t trying to stump people with hard math or obscure trivia. The tasks are things a child could solve. That’s the point — if a task is simple enough for any human but hard for AI, it reveals something real about what’s missing.
The arc of the benchmark series
ARC-AGI-1 (2019): The original benchmark. Static grid tasks where you examine input/output examples and infer a transformation rule, then apply it to a new input. Top AI models struggled for years. OpenAI’s o3 eventually reached roughly 87%, which was close enough to “solving” it that a harder version was necessary.
ARC-AGI-2 (early 2025): Released with a $1 million prize. Harder visual reasoning on the same static grid format. Top frontier models initially scored around 4%. The bar was raised deliberately.
ARC-AGI-3 (2025): A fundamentally different format. Instead of static grids, tasks are interactive. Instead of deducing a rule from examples, you discover rules by taking actions and observing results — like a video game. Humans score 100%. All frontier AI models score 0%.
How ARC-AGI 3 Actually Works
The shift from ARC-AGI-2 to ARC-AGI-3 isn’t about making the puzzles harder in the traditional sense. It’s a change in the type of problem being tested.
Static vs. interactive reasoning
In ARC-AGI-1 and 2, each task gives you a set of input/output grid pairs. You look at the examples, figure out the rule, and apply it to a new case. The format is passive. All the information you need is given upfront — you just have to recognize the pattern.
ARC-AGI-3 is different in a fundamental way. Tasks are presented as interactive mini-environments. There’s no rule description given in advance. You take actions — clicking, moving objects, manipulating the environment — and you observe what happens. The rules reveal themselves through play.
This mirrors how humans actually learn many things. When you pick up a new video game, you don’t read a rulebook before touching the controller. You press buttons, see what happens, and build a mental model of the system through trial and observation.
The structure of each task
Each ARC-AGI-3 task functions as a small, contained game:
- Exploration phase: The solver takes actions to probe how the environment responds and begins building hypotheses about its rules.
- Rule inference phase: Based on observations, the solver constructs a working model of the system’s underlying logic.
- Application phase: The solver uses the inferred rules to achieve a specific goal or produce the correct output.
This approach is sometimes described as “program synthesis through interaction.” Rather than identifying a hidden rule from static examples, you actively test hypotheses in real time, revise them when predictions fail, and narrow in on the correct model of the system.
Why humans find this natural
Humans are exceptionally good at this kind of learning. Give someone a new puzzle game, and within a few moves they’re forming hypotheses and testing them. They can hold multiple competing theories at once, gather evidence, and revise their understanding quickly.
This skill — often called fluid intelligence or inductive reasoning under uncertainty — is something every cognitively typical adult does automatically. It doesn’t require technical knowledge. It requires genuine active reasoning. And it’s precisely what ARC-AGI-3 tests.
Why Every Frontier AI Scores 0%
The 0% result across all frontier models isn’t a fluke, a formatting quirk, or a calibration issue. It reflects something structural about how large language models work.
LLMs are non-interactive by design
Large language models process a static input and generate a static output. They don’t interact with an environment in real time. They don’t update their understanding between steps in a way that reflects genuine learning from experience within a task.
Even in a multi-turn conversation, the model is doing the same thing on each turn: taking the current context window as input and generating a likely next token sequence. It’s inference from a snapshot, not adaptive learning from a live environment.
The memorization problem
A significant part of what makes LLMs appear capable is breadth of coverage. Models trained on vast amounts of data have encountered many problem types and their solutions. When a model answers correctly, it’s often because it’s encountered something structurally similar during training.
ARC-AGI-3 is specifically designed to defeat this strategy. The interactive environments are novel by construction. There’s no prior pattern to recall. You have to actually figure out the rules from scratch — which is the one thing current models can’t do.
Agentic wrappers don’t close the gap
It’s a reasonable intuition: what if you wrap an LLM in an agentic loop and let it take actions inside the ARC-AGI-3 environment? That approach has been tried. The models still score 0%.
The bottleneck isn’t whether the model can technically take actions. It’s whether the model can build a coherent, updating world model from those actions. Current LLMs generate plausible-sounding reasoning, but they don’t genuinely simulate “I tried this, it produced that, therefore the rule is probably X, so now I should try Y.” Each generation step is drawing on patterns in the context window, not grounded inference from lived interaction with the system.
ARC-AGI 3 vs. ARC-AGI 1 and ARC-AGI 2
It’s worth being precise about how these three benchmarks differ, because lumping them together leads to misunderstandings about what’s actually been achieved.
| Feature | ARC-AGI-1 | ARC-AGI-2 | ARC-AGI-3 |
|---|---|---|---|
| Format | Static grid puzzles | Static grid puzzles (harder) | Interactive mini-games |
| Rule delivery | Shown implicitly via examples | Shown implicitly via examples | Discovered through interaction |
| Human score | ~100% | ~100% | 100% |
| Best AI score | ~87% (o3) | ~4% (frontier models) | 0% |
| Core skill tested | Pattern recognition + rule induction | Harder visual reasoning | Interactive learning and adaptation |
| Year released | 2019 | Early 2025 | 2025 |
The jump from ARC-AGI-2 to ARC-AGI-3 is particularly striking. Even though frontier models had made partial progress on ARC-AGI-2, the shift to an interactive format drops performance all the way back to zero.
That’s not just “the tasks are harder.” The type of skill required changed — and it’s a type of skill AI systems don’t have at all.
What ARC-AGI 3 Reveals About Current AI
Beyond the benchmark itself, the results from ARC-AGI-3 are a useful lens for thinking clearly about where AI genuinely falls short.
Pattern matching and understanding are not the same thing
AI systems are extraordinarily good at finding patterns in data. They write code, summarize documents, answer complex questions, generate content, and extract structured information — all at a level that’s genuinely useful. But ARC-AGI-3 puts a precise microscope on the difference between doing those things well and actually understanding a novel system in a generative way.
The benchmark reveals a hard edge: tasks that require active exploration and real-time learning are categorically different from tasks that benefit from prior exposure to similar problems.
The benchmark saturation problem
Many well-known AI benchmarks have been effectively saturated. MMLU, HumanEval, MATH, and others that were once considered strong signals of general capability now see near-human or superhuman scores from frontier models. When that happens, the benchmarks stop being useful — they measure a model’s familiarity with that test distribution, not its general capabilities.
ARC-AGI-3 is designed to resist this dynamic. The interactive, novel-by-construction format makes it much harder to train specifically for the benchmark without achieving the underlying skill.
What genuine agents would actually need
For an AI system to score meaningfully on ARC-AGI-3, it would need something that doesn’t exist in any deployed system today:
- A genuine internal world model that updates based on real-time observation
- The ability to form, test, and revise hypotheses through action
- Efficient use of entirely novel information, not retrieval of familiar patterns
- Meta-learning capabilities — learning how to learn, not just executing known patterns
These are the capabilities ARC-AGI-3 is probing for. The 0% score is confirmation they aren’t here yet.
What This Means for Building with AI Today
ARC-AGI-3 is an evaluation tool, not a product. But understanding what it measures has real implications for anyone building AI systems.
Matching capability claims to actual capability
One practical takeaway from the ARC-AGI results is that it sharpens the distinction between tasks where AI is genuinely useful and tasks where it’s likely to fail. Current models excel at pattern-heavy, text-grounded tasks: writing, summarizing, classifying, coding from context, and extracting structured data. These are exactly the kinds of tasks where large-scale training pays off.
Tasks requiring dynamic, interactive reasoning in genuinely novel environments — the kind ARC-AGI-3 tests — are not where current models belong in your stack.
When building AI agents or automated workflows, being clear about this boundary makes you a more effective builder. You end up designing systems around what models actually do well, rather than hoping they’ll figure out novel reasoning on the fly.
Watching the benchmark for real signals
ARC-AGI-3’s 0% baseline gives researchers and builders a meaningful signal to watch. When a model actually closes the gap on this benchmark, it will likely represent a genuine leap in reasoning capability — not another incremental improvement on a familiar test distribution.
That’s different from most AI announcements, where benchmark gains often reflect training on benchmark-similar data rather than generalization. ARC-AGI-3 is specifically designed to make that kind of shortcut ineffective.
MindStudio gives teams access to 200+ AI models — including all major frontier models — in a single platform, without needing separate API keys or accounts. As model capabilities shift, the platform makes it straightforward to swap models, compare outputs, and update workflows. When ARC-AGI-3 does eventually see progress from any model, the implications for what agents can do will be significant.
You can try MindStudio free at mindstudio.ai. Building a basic agent typically takes 15 minutes to an hour, and the no-code workflow builder handles the infrastructure so you can focus on what the agent actually does.
Frequently Asked Questions
What is ARC-AGI 3?
ARC AGI 3 is the third benchmark in the Abstraction and Reasoning Corpus series, created by François Chollet and Mike Knoop through the ARC Prize Foundation. Unlike earlier ARC benchmarks, which used static grid puzzles, ARC-AGI-3 uses interactive, game-like tasks where the solver must discover rules by taking actions and observing results. Humans score 100% on these tasks. Every frontier AI model currently scores 0%.
Who created ARC-AGI 3?
ARC-AGI-3 was created by François Chollet and Mike Knoop through the ARC Prize Foundation. Chollet is a researcher at Google and the creator of the Keras deep learning framework. He has been developing the ARC benchmark series since 2019 as a framework for measuring genuine general intelligence rather than task-specific performance.
How is ARC-AGI 3 different from ARC-AGI 1 and ARC-AGI 2?
ARC-AGI-1 and ARC-AGI-2 both use static grid puzzles — you see examples and infer a transformation rule. ARC-AGI-3 is interactive. Instead of being given examples to analyze, you take actions in a mini-game environment and observe what happens. The rules emerge from interaction, not from pattern recognition on static inputs. This shift completely breaks current AI approaches, dropping performance from partial scores on earlier benchmarks to 0% on ARC-AGI-3.
Why do AI models score 0% on ARC-AGI 3?
Current large language models are fundamentally non-interactive. They process static inputs and generate outputs based on trained patterns. ARC-AGI-3 requires real-time learning from action and observation — forming and revising hypotheses as you explore. This is a type of reasoning LLMs don’t perform. Wrapping them in agentic loops doesn’t solve this, because the bottleneck is the model’s ability to genuinely update a world model through experience, not whether it can technically execute actions.
Is there a prize for solving ARC-AGI 3?
The ARC Prize Foundation has structured significant prizes around the ARC benchmark series. ARC Prize 2025 offered $1 million for meaningful progress on ARC-AGI-2. Prize structures for ARC-AGI-3 are set by the Foundation and are designed to incentivize genuine progress on machine reasoning — not benchmark-specific tuning. Check the ARC Prize Foundation’s official site for current prize details.
Does ARC-AGI 3 prove that AI will never reach human-level reasoning?
No. ARC-AGI-3 shows that current AI systems lack a specific capability — interactive, inductive reasoning in novel environments. It doesn’t make any claim about future systems. In fact, the benchmark exists because prior versions got solved or nearly solved. The ARC series is explicitly designed to stay ahead by testing capabilities that matter for general intelligence, not just the ones current systems happen to be good at. The 0% score is a current measurement, not a permanent ceiling.
Key Takeaways
- ARC AGI 3 uses an interactive, game-like format — unlike earlier ARC benchmarks, tasks require discovering rules through action and observation, not pattern recognition on static examples.
- Humans score 100%. All frontier AI models score 0%. This gap reflects a structural difference in capability type, not just task difficulty.
- Current LLMs are non-interactive by design. Real-time adaptive learning from a live environment is outside the architecture of today’s models, even when wrapped in agentic loops.
- ARC-AGI-3 is designed to resist benchmark gaming. Novel-by-construction interactive tasks can’t be shortcut through training on similar data.
- For practitioners: Understanding these capability boundaries helps you build more reliable AI systems — deploying models where they’re genuinely strong, and watching the ARC-AGI-3 score closely for signals of genuine reasoning breakthroughs.
If you want to build AI workflows with today’s best available models, MindStudio lets you access 200+ models and deploy agents without writing code. Start free at mindstudio.ai.