Study Finds Top AI Experts Badly Underestimated Pace of Progress
Tech

Study Finds Top AI Experts Badly Underestimated Pace of Progress

TechNews Editorial
TechNews EditorialSep 25, 2026 · 2 min read
Share

Top university experts, highly cited artificial intelligence researchers, and veteran economists significantly underestimated recent progress on benchmarks and adoption metrics. This finding comes from an interim report published by the Forecasting Research Institute.

Since mid-2022, the organization has gathered predictions on artificial intelligence progress across multiple projects. Its participant pool includes senior specialists. The first round of the Longitudinal Expert AI Panel featured 339 experts. This group included 76 computer scientists, 76 industry professionals, 68 economists, and 119 policy specialists.

The computer scientists included 30 professors at top-20 institutions and 10 of the 200 most-cited authors in the field. The panels also featured superforecasters, who are generalists with a proven track record of making accurate predictions.

Artificial intelligence reached major milestones years ahead of initial forecasts. The largest gap involves mathematics. The technology achieved gold-medal performance at the International Mathematical Olympiad in July 2025. That accomplishment arrived five years before the median expert forecast and ten years before the median superforecaster prediction.

Those specific predictions were gathered in 2022 before the launch of ChatGPT. However, the report states that the pattern held true for later predictions as well. Based on their forecasts, experts assigned an average probability of 24.6 percent to the benchmark results that actually occurred. Superforecasters assigned just 9.7 percent.

For gold-medal performance at the Math Olympiad, those figures dropped to 8.6 percent and 2.3 percent respectively. Artificial intelligence may have also solved a Millennium Prize Problem. It remains unclear whether that solution fully meets all the required evaluation criteria.

In a survey conducted in August and September 2025, experts placed the median odds of such a solution by the end of 2027 at just 10 percent. Superforecasters put those odds at 5.4 percent. Economic forecasts were also far too conservative.

Experts estimated the median for the highest annual recurring revenue of any artificial intelligence company at the end of 2026 at $20 billion. Economists predicted $16 billion, while superforecasters estimated $25 billion. The research organization cites roughly $100 billion for Anthropic in September 2026 as a figure that has likely already been reached.

Not every forecast ran too low. Biosecurity experts predicted that 22.5 percent of participants using a language model would successfully complete biological lab tasks. Virologists expected 40 percent, and superforecasters predicted 16.2 percent. In a controlled trial, only 5.2 percent succeeded using a language model with internet access, compared to 6.6 percent using the internet alone.

Experts may have also overshot regarding self-driving cars. Their median forecast for the share of autonomous United States ride-hailing trips in 2027 stood at 7.3 percent. Meanwhile, a large language model projection placed that figure at 2.5 percent.

Respondents are actively revising their expectations upward. Among participants who completed both surveys, the average probability assigned to artificial intelligence becoming a technology of the century rose from 31 to 36 percent for experts. For superforecasters, that figure climbed from 28 to 35 percent over nine months.

Going forward, the Forecasting Research Institute will highlight a subsample of respondents who anticipate very rapid progress through 2040. The organization will also publish continuously updated large language model forecasts alongside human predictions. The institute notes that underestimates become obvious as soon as reality overtakes a prediction, while overestimates only become clear after a deadline passes.

Newsletter

Get the best AI & tech news daily

A concise daily digest. Unsubscribe anytime.

We use your email only to send this newsletter.

Related Stories