When AI Finds Mistakes the Experts Missed: Inside Science's New Co-Author
AI models are catching errors in physics benchmarks, math papers, and decades-old conjectures. Real cases show what this means for research.
What happens when AI scores better than the answer key?
A group of researchers built a theoretical physics test called TheoPhysEval, made up of 50 unpublished, research-level problems, the kind a world expert might need a full day to solve. When they ran a frontier AI model against it, the model appeared to fail several questions. But when the researchers checked with the physicists who wrote those problems, they found the official answers were wrong, not the AI’s. After correcting the answer key, the model’s score rose to 49 out of 50. This is the clearest sign yet that AI isn’t just passing tests scientists design. In some cases, it’s catching mistakes the test designers themselves missed.
TL;DR
- A physics benchmark called TheoPhysEval, built by researchers from Harvard, UT, Google DeepMind, and other institutes, used 50 brand-new, unpublished problems specifically so the AI couldn’t have seen the answers in training.
- When an advanced model disagreed with the official answer key, experts rechecked their own solutions and found the AI was right and the key was wrong, pushing the model’s score to 98% after corrections.
- A similar pattern showed up in Frontier Math, a benchmark built by Epoch AI with professional mathematicians: an AI-assisted review flagged fatal errors in roughly a third of the problems.
- A separate study had an AI model scan nearly 800 math papers hunting for counterexamples that disprove central claims, finding major errors in about 6% of older papers and only 1.5% in recent ones written after AI use became common.
- Physicist Gabriele Carcassi described an AI solving a conjecture in two minutes that he’d been stuck on for a year and a mathematician had struggled with for six months.
- OpenAI reported a model producing a proof for the cycle double cover conjecture, a networks problem open for around 50 years, by running dozens of AI processes in parallel for under an hour.
- The pattern raises a genuine measurement problem: if the people writing tests and papers are sometimes the weakest link, how do you grade a system that might know more than the graders?
Auditable. Scalable.
Why does it matter that AI found errors in a physics test?
The TheoPhysEval story matters less because of the final score and more because of what it reveals about how we’ve been measuring AI capability. The test was deliberately built from unpublished problems so models couldn’t rely on memorized answers scraped from the internet. That’s a serious design choice meant to measure actual reasoning, not recall.
When the model’s answers conflicted with the official key, the default assumption should have been that the model was wrong. Instead, researchers went back to the physicists who wrote the problems, and in several cases the original solutions contained subtle errors, the kind of sign flip or dropped term that can quietly unravel a page of calculation. Once those were fixed, the model’s score jumped from an already strong result to 49 out of 50.
That has a knock-on effect on every other model ever tested against that same answer key. Any system that got the right answer on those flawed questions would have been marked wrong, meaning previous scores on this kind of benchmark may have understated real capability. Researchers did note the model still failed some problems, especially without access to its tools, and one problem went unsolved by every model tested. So this isn’t a story about flawless AI. It’s a story about a test that turned out to be harder to grade correctly than to solve.
Is this happening outside of physics too?
Yes, and arguably on a larger scale in mathematics. Frontier Math, a benchmark from Epoch AI built with professional mathematicians, was designed to resist AI for years; problems can take human experts hours or days. When it launched in late 2024, top models solved less than 2% of it. Less than two years later, an AI-assisted review of the benchmark flagged fatal errors in about a third of its problems, and researchers said they believed most of those flags were valid.
OpenAI researcher Noam Brown connected the two stories directly, suggesting physics benchmarks might be following the same trajectory math benchmarks already have: tests built to be hard for AI are turning out to have embedded human error that AI is uniquely suited to catch.
Everyone else built a construction worker.
We built the contractor.
One file at a time.
UI, API, database, deploy.
A separate research effort pushed this further by pointing AI at actual published work rather than benchmarks. A team led by Jasper De Coninck and Martin Vechev used a GPT model to scan nearly 800 math papers, extracting each paper’s central claims and searching for counterexamples, specific cases that would disprove the claim. The process cost around $2,500 in computing for hundreds of papers, far cheaper than paying a human expert to review even a handful. The model found major errors (meaning the core result was false) in about 6% of papers from before August, and minor fixable errors in another 30%. Researchers estimated close to 90% of the AI’s major-error flags held up under human review. Papers from more recent months, when over a quarter of authors disclosed using AI themselves, had a much lower major-error rate of about 1.5%. The researchers were careful not to claim causation, but the pattern suggests AI may be helping prevent errors, not just catching them after publication.
What does this look like for a scientist actually using it?
Benchmarks are useful, but the more striking evidence comes from individual researchers. Physicist Gabriele Carcassi at the University of Michigan described a conjecture about topologically convex spaces that he had been unable to prove for a year, despite a mathematician helping him on and off for six months. He gave the problem to an AI model and had an answer in two minutes, using a technique (the tube lemma) he said he hadn’t even known existed.
Carcassi’s own framing is worth sitting with: the AI isn’t a genius, and he said you have to check everything it produces, but “it also knows everything.” That combination, unreliable in isolated spots but unmatched in breadth, is what makes it useful as a research collaborator rather than a replacement for one. It can suggest an approach from a field a researcher never studied, something no single human collaborator is likely to have on hand.
A bigger example came when OpenAI reported that one of its models produced a proof for the cycle double cover conjecture, a problem about network structures that had been open for roughly 50 years. The model reportedly used a swarm of around 64 parallel AI processes and finished in under an hour. Mathematician Thomas Bloom reportedly called it a nice, short, elementary proof that could plausibly have been found decades earlier, and suggested the real reason humans missed it is that people give up after the obvious approach fails, while an AI system can grind through small variations indefinitely.
That story isn’t uniformly positive. When OpenAI later released a batch of AI-generated math papers at once, the mathematics community’s reaction was mixed: some called it significant, others were frustrated, and a handful of papers had to be retracted for errors. The technology is producing real results and real noise at the same time.
How do you test something that might be smarter than the test-writer?
This is the uncomfortable question underneath all of these stories. The standard way to evaluate AI is to give it a test built by human experts with a human-verified answer key. That system assumes the humans are the ground truth. But when a model can out-reason the people who wrote the key, as happened with TheoPhysEval, or when a third of a flagship benchmark’s problems turn out to be flawed, as happened with Frontier Math, the grading process itself becomes the bottleneck.
Benchmark saturation (tests getting solved faster than expected) used to be the main problem: a new test would arrive, models would beat it within months, and researchers would build a harder one. Now there’s a second-order problem. It’s not just that tests are too easy. It’s that human-authored answer keys may simply contain mistakes that only become visible once a sufficiently capable system pushes back on them.
One proposed fix is leaning on problems that can be checked automatically rather than by human judgment: code that either runs correctly or doesn’t, or formal math proofs verified step by step by proof-checking software with no ambiguity. That removes the human grader from the loop entirely for certain problem types, though it only works where a problem can be fully formalized, which isn’t true of most open research questions.
Frequently Asked Questions
What is TheoPhysEval?
It’s a physics benchmark made of 50 unpublished, research-level theoretical physics problems, built by researchers from institutions including Harvard, UT, and Google DeepMind. It was designed to be hard enough that experts need about a day per problem, and new enough that AI models couldn’t have memorized the answers.
What is Frontier Math?
Frontier Math is a benchmark built by Epoch AI with professional mathematicians, designed to contain extremely difficult problems that would resist AI progress for years. An AI-assisted review later found fatal errors in roughly a third of its problems.
What is the cycle double cover conjecture?
It’s a longstanding open problem in graph theory concerning networks, open for around 50 years. OpenAI reported that one of its models produced a proof for it using a large number of parallel AI processes running in under an hour.
Does this mean AI is now smarter than human scientists?
Not exactly. The researchers and scientists quoted in these cases are consistent on this point: the models still make mistakes and need checking, but they bring extremely broad knowledge and can connect techniques across fields faster than most individual humans can.
Why does it matter if an answer key is wrong?
Because every score measured against a flawed answer key is unreliable. If a model is marked wrong for giving the actually-correct answer, it understates real capability, and it means the benchmark itself needs re-auditing before it can be trusted for comparing models.
