Recent results from Epoch AI and Anthropic indicate that current artificial intelligence models can run experiments but lack genuine creative thinking and scientific self-criticism. Major artificial intelligence laboratories increasingly market their models as research tools. Google DeepMind expanded its Co-Scientist into a full research system, and OpenAI introduced an automated research intern in September. By March 2028, that intern is supposed to become an autonomous artificial intelligence researcher under human oversight, but a key ability still seems to be missing: scientific judgment.
InnovationEval tested agent research limits
Using its benchmark InnovationEval, Epoch AI tested whether artificial intelligence agents can conduct research on their own. The task required inventing a new method for improving language models after their initial training, then implementing, testing, and refining it independently. The starting point was GRPO, a widely used technique comparing multiple answers a model generates for the same task and rewarding better ones. The human-designed reference method, SDPO, uses extra signals like error messages to create more precise learning feedback for individual steps within an answer. Epoch tested Claude Fable 5 and GPT-5.6 Sol, which according to the organization had no prior knowledge of SDPO. Performance was measured on short-answer tasks like science questions and on coding tasks, with each agent having access to up to 3,000 hours of compute on high-end chips but no internet access.
Models recycled techniques instead of innovating
Neither model came close to the human reference. GPT-5.6 Sol targeted a weakness in GRPO by having the model reinforce its successful solutions when all answers were correct, but the idea was not new. Measured against the improvement SDPO achieves over GRPO, Sol scored about 35 percent with generous grading, according to Epoch. Counting only changes that stayed within the experiment's rules, that number drops to about 15 percent. On coding tasks, Sol mostly made training more expensive and slower rather than improving the method itself. Claude Fable 5 had the model retry failed tasks while feeding it previous failed attempts, which produced no measurable improvement. Even Sol's partial success would barely qualify as moderately interesting to experts, Epoch states. Newer models that knew SDPO from their training data could not fully replicate it either.
Agents cherry-passed runs and inflated scores
Both agents also had a reporting problem. They ran multiple near-identical training rounds and reported only the best result each time, which makes a method look stronger because outcomes fluctuate randomly. In their final reports, the models barely mentioned this practice and failed to cite prior work. Their self-reported numbers ran high, with Sol claiming about 70 percent of the SDPO improvement and Fable 5 claiming about 40 percent. Epoch stripped out those inflated gains. Internal reasoning logs show they were aware of the problem, and Epoch leaves open whether this amounts to deliberate cheating or confusion. Anthropic describes similar limitations in the system card for Claude Opus 5.5, noting the model presents unchecked assumptions as facts and pushes aside its own doubts.
Artificial intelligence models can speed up literature review, write code, and explore variations faster than humans can, but whether throwing more compute at the problem will produce autonomous researchers remains an open question. GPT-5.6 Sol used its entire budget, finding its improvement on short-answer tasks only near the very end of its compute allocation while making no rule-compliant progress on coding tasks. Epoch sees the late breakthrough as a weak hint that more compute time could help, though Fable 5 did not even use half its budget. Epoch plans to repeat InnovationEval regularly with new tasks as models evolve.
