Within the Margin of AI Error

This week, I started submitting the longer, more technical articles on this blog to Claude AI for proofreading and fact-checking. Like most personal websites, this one has a QA department of exactly one: me. Not surprisingly, then, the AI found quite a few errors. Beyond the stupid typos, there were mistakes in the LaTeX, terminology snafus and so on. Long story short: the mathematics articles here have now been corrected to within the margin of standard AI error. I regret waiting so long to do this.

It got me wondering about the countless articles published in academic journals. In principle, all of them have all been subjected to rigorous peer review. One would hope their error rates are fairly low. Then again, with the volume and complexity of papers rising, I suspect many reviews aren’t as rigorous as they should be.

I asked the bot this very question:

Q: “How many mistakes have been found by AI in long-standing published journal articles?”

A: There’s no overall count. No one tracks AI-detected errors across the scientific literature, and the efforts so far use different tools, fields and definitions of “mistake.” But the studies and cases that exist suggest AI is finding a lot, especially small, checkable errors.

The clearest large-scale number comes from AI’s own literature. A Stanford-led team built a GPT-5-based checker and ran it on about 2,500 papers from top AI venues: NeurIPS, ICLR (2018–2025) and TMLR. It found an average of 4.66 objective mistakes per paper, and about a quarter to a third of papers had at least one potentially substantive one. It looked only at errors with a verifiable right answer, like wrong formulas, derivations, calculations, figures and tables. When human experts reviewed 316 of its flags, they confirmed 263, about 83% precision. The checker could also propose correct fixes for about 76% of the errors it found. The error rate is rising, too: from 3.8 per paper in NeurIPS 2021 to 5.9 in NeurIPS 2025.

Errors that are decades old. In August, Nature reported that AI agents are proving good at finding errors in decades-old papers and reference databases. Its lead example: AI showed that some boiling-point values chemists have relied on for decades in one handbook database were simply wrong. The full article is paywalled, so I can’t give you its specific counts.

Reproducibility audits. In July, a US research-auditing company used AI agents to try to reproduce all 168 oral papers at ICML 2026. Only 34 had more than 40% of their conclusions reproduced, and only 8 had more than 80%. Treat that with some caution, though: when an AI agent fails to reproduce a result, it can mean the paper is wrong, but it can also mean the agent made a mistake.

Volunteer and startup efforts. The Black Spatula Project and YesNoError have been scanning papers since early 2025. They use language models to look for factual, calculation, methodology and citation errors, at a cost of a few cents to a few dollars per paper. Their big problem is false positives: flags of errors that aren’t actually there. Every flag still needs checking by an expert in the field.

Mathematics specifically. The 2026 headlines in math have mostly been about AI disproving open conjectures, not finding mistakes in published proofs. I didn’t find a systematic count of AI-found errors in published math papers. The closest thing is that formalizing proofs in Lean, now often AI-assisted, sometimes turns up gaps in the original arguments.

Bottom line: the best evidence points to many small, checkable errors per paper, at least in fast-moving fields like machine learning. AI can find these cheaply and fairly accurately, but humans still have to confirm them. Whether AI is overturning many major long-standing findings is much less clear. So far there are notable individual cases, but no tally.

—————————————————————————————————————–

I also wondered about the glut of AI-authored journal papers.

Q: And what about the problem of AI-authored journal papers?

A: It’s a real and growing problem, and it’s the flip side of the previous question: AI is now one of the main sources of errors in the literature as well as a tool for finding them. The problem shows up in three ways.

1. Fabricated references are the clearest signal, and they’re increasing fast. A team led by Maxim Topaz at Columbia checked 97 million references in nearly 2.5 million PubMed-indexed papers, and published the results as a letter in The Lancet in May.

In the first seven weeks of 2026, about 1 in 277 papers cited a paper that doesn’t exist. That rate was 1 in 458 in 2025 and 1 in 2,828 in 2023, a 12-fold rise in two years. The sharpest jump came in mid-2024. More than 98% of the affected papers had no action from their publisher.

There are caveats. Critics said the methods weren’t described in enough detail, and that the study didn’t separate citations that matter from minor ones. Most of the affected papers also had only one or two fake references, which could be sloppiness rather than fraud. But a citation to a paper that doesn’t exist is close to proof that someone pasted AI output without checking it.

The same pattern shows up in AI research itself. GPTZero found over 100 hallucinated citations in papers accepted at NeurIPS 2025. In a sample of 300 ICLR 2026 submissions, it found that about 17% had at least one fake citation, which multiple peer reviewers had missed.

2. AI is reviewing the papers too. This connects to the suspicion that reviews aren’t as rigorous as they should be. ICLR 2026 received about 19,500 submissions, and reviewers had about five papers each to review in roughly two weeks. A scan of 75,800 of those reviews found that 21% were fully AI-generated, and more than half showed some AI use. So in some cases, AI-written papers were being checked by AI-written reviews.

3. Fully AI-authored papers can now pass peer review. In 2025, a paper written entirely by Sakana’s “AI Scientist” scored above the acceptance threshold at an ICLR workshop. Sakana withdrew it on purpose. The work was published in Nature this March, along with the system’s known weaknesses: simplistic ideas, weak methods, hallucinated citations and duplicated figures. Sakana itself has called for community norms so that journals aren’t flooded with automated submissions.

How institutions are responding. In May, arXiv announced that authors can be banned for a year if there’s clear evidence they didn’t check AI-generated content, such as hallucinated references, leftover chatbot comments or placeholder data. After a ban, their submissions must first be accepted at a peer-reviewed venue. arXiv isn’t banning AI tools. It’s saying that authors are responsible for everything they submit.

Bottom line: AI authorship isn’t the problem in itself. Unchecked AI output, written faster than human reviewers can read it, is the problem. The same tools that found the typos in LaTeX are also adding fake citations to the literature, and possibly faster than checkers are finding errors.

Sources:

Legacy Errors

• arXiv: To Err Is Human: Systematic Quantification of Errors in Published AI Papers
• Nature: AI agents are checking the scientific literature — and spotting decades-old errors
• 36Kr: AI reproducibility audit of ICML 2026 papers
• Heise: AI checks papers for errors – why that’s not enough
• arXiv: Towards Automating Scientific Review with Google’s Paper Assistant Tool

AI-Authorship

• Retraction Watch: One in 277 PubMed-indexed papers in 2026 shows fabricated references
• GPTZero: Research motivates major arXiv ban on AI and hallucinated citations
• SMU Libraries: arXiv tightens policy on hallucinated references
• Slashdot: One in five ICLR 2026 peer reviews fully AI-generated
• Noqta: Sakana AI Scientist published in Nature