GPT-6-Astra solved 49 of 50 unpublished theoretical physics problems in a new benchmark called TheoPhysEval. Thirty-five researchers from institutions including the International Centre for Theoretical Sciences (ICTS), Harvard University, KU Leuven, and Google DeepMind built the test to measure how well AI agents handle research calculations.
The 50 problems have verifiable answers and span eleven subfields, from fluid flows and biological processes to particle physics and cosmology. According to the authors, a leading specialist could solve a typical problem in about a day or less without AI. Someone in a neighboring field would take much longer. The benchmark leaves out major open questions and focuses on what the authors call "turn-the-crank" work: applying known methods to a changed question. Contributors often altered values in calculations from their own recent papers so an old result would not give away the answer.
The agents solved demanding calculations
One black hole problem required finding an older paper, adapting its equations from five-dimensional to four-dimensional spacetime, and writing a program to compute the result. Coordinator Suvrat Raju, a co-author of that paper, needed about a day. The authors estimate that, before modern agents, only a handful of experts had the necessary mix of physics and computational experience. Even they would have needed several days.
The team tested commercially available agents without human hints. Each agent took the test twice: once with web search and once without internet access. GPT-6-Astra, running in OpenAI's Codex environment, led with 98 percent. Its full benchmark run cost just under $116, averaging under $3 per problem. Anthropic's Opus 5.5 in Claude Code scored 92 percent. Google's Gemini 3.8 Flash reached 68 percent, while the open-weight DeepSeek-V4.1-Flash and GLM 5.3 scored 54 and 42 percent.
Some weaker agents made errors an expert would likely catch quickly. In the web search run, Gemini and GLM used formulas from the older black hole paper without adapting them from five dimensions to four. Web access made little difference to the leaders: Astra and Opus each scored 92 percent without it. Because many questions depend on papers the agents were not given, the authors interpret those results as evidence that leading models have retained substantial parts of the theoretical physics literature.
Answer checks changed some results
With web search, Astra missed only one problem, which every model failed. It asked for an equation producing a given curve. The experts' solution uses special mathematical forms that qualify as the requested differential equation only in a broader sense. The authors acknowledge that the question may seem "unfair," but say it reflects the task of inferring physics from imprecise data.
Follow-up checks with experts also found subtle errors in several reference answers. After those were corrected, Astra's answers to those problems were right. The authors say grading errors may help explain why agents scored better on TheoPhysEval than on benchmarks such as CritPt, despite their view that their problems are harder on average. A recent study likewise found incorrect grading in some theoretical physics benchmarks. The problems and test setup are available as a dataset on Hugging Face.
Read nextNvidia pledges $250 million in compute for UK AI safety researchThe authors say research needs more than answers
The authors say some leading models may already outperform top researchers on specific calculations. They point to mathematics, where AI agents have solved important open problems. According to OpenAI, that includes the Navier-Stokes Millennium Prize Problem concerning fluid-flow equations. The authors say theoretical physics has not seen a comparable AI breakthrough.
They argue that calculation comes late in research. First, a physicist must build a theory, choose the "right" question, and sometimes switch to a simpler but equally interesting one. Current agents struggle with those choices and appear to lack the shared, though far from unanimous, sense of "taste" physicists use to judge interesting problems. The authors say these observations cannot be quantified and are not a forecast, though they think agents could conduct top-level research independently in the near future.
The results also affect training: problems once suitable as first research projects for PhD students may now serve mainly to help them work through the material. The authors warn that harder AI-assisted work could demand much larger budgets, shifting a field that has historically required far less funding than experimental research. With mathematics already debating access, including around the Leiden Declaration, they call for theoretical physicists to discuss equal opportunity before more capable agents reshape the field.
