AI and Math: From Broken Arithmetic to a Millennium Prize Problem
How AI models went from failing basic arithmetic in 2023 to winning IMO gold and solving a Millennium Prize problem by 2025.
What’s the fastest way to summarize AI’s progress in math?
In 2023, leading language models routinely botched four-digit addition. In 2024, they struggled with most problems on a high school math competition. By 2025, a model won a gold medal at the International Math Olympiad, and a few months later, OpenAI reportedly solved a Millennium Prize problem in 88 hours. That’s roughly two years to go from arithmetic errors to cracking one of the hardest unsolved problems in mathematics, the kind that earns a permanent place in math history and, under the Clay Mathematics Institute’s program, a $1 million prize.
TL;DR
- AI models went from failing basic four-digit arithmetic in 2023 to winning gold at the International Math Olympiad in 2025, a jump that took less than two years.
- A frontier model reportedly solved a Millennium Prize problem in about 88 hours, a category of problem that has stumped professional mathematicians for decades.
- The progress is concentrated in verifiable math, meaning proofs, counterexamples, and well-scoped problems where an answer can be checked mechanically, not in generating new mathematical theory or asking novel questions.
- Researchers like Anthropic and OpenAI report that AI is already accelerating AI research itself, with internal tools handling a growing share of R&D work compared to a year ago.
- The argument for faster AI progress rests on three multipliers: models work at roughly 10 times human speed, each one could reach the skill level of top human researchers, and labs can run many of them in parallel.
- Skeptics point out that solving conjectures is different from inventing new fields of math, so the Millennium Prize result says more about search and verification skill than creative mathematical insight.
- Estimates for when AI matches the best human AI researchers range from 2027 to after 2032, with substantial uncertainty baked into every forecast.
Without shadow IT.
Why did math become the test case for AI capability?
Mathematics offers something rare: a domain where answers can be checked with total certainty. A proof is either valid or it isn’t. A counterexample either breaks a conjecture or it doesn’t. That makes math an unusually clean testing ground for measuring whether AI systems are actually getting smarter, as opposed to just getting better at sounding confident.
This matters beyond math itself. The same verification property shows up in competitive programming, formal theorem proving, and increasingly in AI research itself, where you can measure whether a new training technique actually improves a model’s benchmark scores. Progress in math has become a leading indicator for progress in these adjacent, highly verifiable domains.
How fast has the timeline actually moved?
The sequence, as it played out publicly: models that couldn’t reliably add four-digit numbers in 2023, models that could barely touch high school competition problems in 2024, a gold-medal performance at the International Math Olympiad in 2025, early solutions to niche open problems in research mathematics later that year, and then a Millennium Prize problem solved in roughly 88 hours a month after that.
Each of these represents a different tier of difficulty. The IMO is a known, bounded competition with problems designed for exceptionally talented high schoolers working under time pressure. Open research problems are a different animal entirely: no guaranteed solution exists, no one has graded a rubric in advance, and professional mathematicians may have tried and failed for years. A Millennium Prize problem sits at the top of that hierarchy, a list of seven problems identified by the Clay Mathematics Institute as among the most important unsolved questions in mathematics, only one of which (the Poincaré conjecture) had been resolved before this round of AI progress.
Watching a system move through all four tiers in roughly two years is what changed minds about how close AI might be to expert-level performance across other technical fields.
Is this the same as AI being creative or doing original research?
Not quite, and this distinction matters. The progress described above is concentrated in what’s sometimes called “search and verification”: trying many candidate proofs, counterexamples, or approaches, and checking which ones hold up. That’s a persistence and pattern-matching problem, not necessarily a creativity problem.
Critics, including mathematicians who’ve written about this shift, note that solving a well-posed conjecture is different from inventing a new mathematical field, the way group theory or topology opened up entirely new ways of thinking about structure. Current systems show little evidence of asking the kind of open-ended question that redefines what a field of math is even about. They’re excellent at grinding through a known space of possibilities with superhuman persistence, not at deciding which questions are worth asking in the first place.
But the counterargument carries weight too: a system that can solve a Millennium Prize problem without that deeper creative capacity is still solving a problem that defeated every working mathematician who tried before it. That’s a meaningful capability regardless of how it got there.
Why does this matter for AI research specifically, not just math?
The mechanics that let AI grind through math problems translate unusually well to AI research itself. Training a better model is not primarily about discovering new theoretical frameworks. It’s about testing combinations of existing techniques (data filtering methods, attention mechanisms, optimizer tweaks) to see empirically what makes models train more efficiently or perform better. That is closer to brute-force combinatorial search than to inventing calculus.
The transformer architecture, the foundation of modern large language models, is a commonly cited example. Researchers active in the field before 2017 had already identified the individual ingredients, attention mechanisms, residual connections, and other components, as useful. What took skill was combining roughly half a dozen of these known pieces in the right configuration, which required extensive experimentation rather than a single theoretical leap. That kind of persistent, combinatorial problem-solving is exactly what current systems have shown disproportionate skill at in mathematics.
This is part of why organizations building frontier models report measurable upticks in internal AI usage for research tasks. Independent evaluators who’ve spent time embedded inside AI labs have reported that the pace of progress on well-scoped engineering tasks has roughly doubled year over year, with much of that acceleration attributed to heavier internal use of AI coding and research tools. Separately, usage data shared by major labs shows a sharp rise, within a matter of months, in how much internal compute the heaviest users were spending on AI-assisted research work.
How much faster could AI research get if this trend continues?
The case for a significant speedup rests on three compounding factors. First, AI systems can already operate at a much higher throughput than human researchers, running continuously rather than during a standard workday. Second, if models reach the skill level of the best human AI researchers, that matters disproportionately: a small number of top researchers are believed to drive a large share of a lab’s overall algorithmic progress, which is part of why top AI talent commands extraordinary compensation. Third, AI research includes a lot of modular, independently checkable work, like tuning data pipelines, writing kernels, or designing training environments, that lends itself to running many parallel experiments rather than one sequential effort.
None of these multipliers compound without limits. Diminishing returns show up quickly: more researchers, human or automated, do not multiply output linearly, and experiments are still bottlenecked by available compute. But even accounting for steep diminishing returns, the combination of speed, skill, and parallelism is the basis for forecasts that AI could match top human AI researchers within the next few years rather than the next decade.
Is this prediction reliable?
Estimates vary widely even among people closely tracking this space. Some place the milestone (AI matching the best human AI researchers) as early as next year. Others see a meaningful chance it doesn’t happen until after 2032. The uncertainty comes down to how much of AI research resembles the clean, verifiable problems where AI has excelled, versus how much depends on harder-to-measure judgment, like betting on a research direction years before it pays off.
What’s not in dispute is the underlying pattern: in every domain tested so far where answers can be verified and models can practice at scale, AI performance has moved from below-average to superhuman within a few years. Math is simply the clearest, most public example of that pattern playing out in real time.
Frequently Asked Questions
What is a Millennium Prize problem?
It’s one of seven mathematics problems identified by the Clay Mathematics Institute as among the most significant unsolved questions in the field, each carrying a $1 million award for a verified solution. Only one, the Poincaré conjecture, had been solved prior to this recent wave of AI progress.
Did an AI model actually win an International Math Olympiad gold medal?
Yes, according to reports in 2025, a frontier AI model achieved a gold-medal-level score on IMO problems, a sharp jump from 2024 when models could barely solve any competition-level problems.
Does solving math problems mean AI can do original scientific research?
Not directly. Current systems excel at well-scoped, verifiable problems where answers can be checked mechanically, like proofs and counterexamples. There’s little evidence yet that they can identify genuinely new research questions or open new fields of inquiry the way breakthrough mathematicians historically have.
Why does math progress matter for AI research automation?
Because machine learning research itself involves a lot of verifiable, combinatorial work: testing which algorithmic tweaks improve training efficiency or model performance. That’s structurally similar to the search-and-verify math problems AI has recently mastered, which is why some researchers see math progress as a signal for how fast AI research itself could get automated.
When do experts think AI will match top human AI researchers?
Estimates range from as early as 2027 to after 2032, depending on assumptions about how much of AI research depends on verifiable, practicable skills versus harder-to-measure judgment and long-term strategic bets.
