AI solves more research-math problems; attribution still trails correctness
First Proof’s new ten-problem evaluation found nine correct solutions from ProofCouncil and seven from direct GPT-6 Astra, with fewer submissions meeting the citation standard.
First Proof tested five systems on ten previously unpublished, human-solved research-mathematics problems.
Thirty experts assessed anonymized submissions. ProofCouncil produced nine correct solutions; direct GPT-6 Astra produced seven. The results were published on October 14. [1][2]
Correctness and scholarly completeness are separate tests. Seven of ProofCouncil’s ten submissions had adequate citations, compared with four of Astra’s ten. These counts do not mean that all adequately cited submissions were also correct. Exposition remained a weakness. [1][2]
The evaluation gives mathematicians a concrete reason to experiment with AI-generated approaches while checking every argument and its intellectual debts. It remains a small, selected problem set, with different surrounding software and compute budgets. The published problems, solutions and referee reports are more informative than treating the leading score as a general probability that an AI proof is sound. [2]
Sources & details 2 sources
Discuss in your voice app
Take the reporting, visual explanations and original sources into a conversation. Copy the brief, paste it into your preferred assistant, then start its voice mode.
Paste the brief into your voice app to discuss this reporting.Reporting sources
- First Proof · Third-batch results
October 14 announcement by the evaluation organizers; ten problems, 30 reviewers, separate correctness and citation assessments.
- First Proof · Full evaluation report
Original 46-page report, including methods, system differences, grading and limitations. This is the same evaluation, not an independent replication.
Images & figures
Correctness and adequate attribution are separate criteria, each assessed across ten submissions.
Euangelion · Original data figure; First Proof results · Source · Usage terms · Download figure