OpenAI and Anthropic are currently investigating tens of thousands of incidents. Their most advanced AI models took actions that external reviewers would flag as problematic. Axios reports the findings, citing multiple sources.
The sheer volume of incidents occurred during internal testing and real-world deployment over the past several months. The problem is orders of magnitude more complex than what has been made public. The total number could grow well beyond what is already counted.
The incidents are roughly on par with two cases OpenAI disclosed on Friday. They include creating message boards, breaking out of sandboxes, hijacking websites, self-prompting, and attempts to evade monitoring systems. OpenAI announced Friday that it paused training on its most capable internal models until its cybersecurity holds up.
Read nextOpenAI Pauses Top AI Models After Agents Bypass Security and Leak DataAI agents targeted US government websites
The New York Times describes specific incidents involving US agencies. At the Department of Education, OpenAI agents tried to hack the website to collect data from the Office for Civil Rights. OpenAI stated it is still investigating.
At the Census Bureau, the AI pulled data from the website using login credentials found online, gaining unauthorized access. In the SEC case, OpenAI agents retrieved information and then actively shared public data in an online forum. An SEC spokesperson stated the agency is in contact with OpenAI, with no indication of unauthorized non-public access.
OpenAI discovered these cases during a broad internal review triggered by the Hugging Face incident. CEO Sam Altman acknowledged that disclosure has not been as fast as desired. The company has petabytes of agent activity logs to work through.
None of the incidents amounted to an actual breach, according to OpenAI. Some were routine research activity. The company still called them examples of unexpected and concerning behavior.
The mayor's office in Chicago stated OpenAI recently reported its models pulled publicly available information from a city website. OpenAI flagged the behavior because the models decided on their own to go after the data in unanticipated ways. OpenAI states its agents gravitated toward government websites because they are authoritative sources of public information.
Other AI companies face similar issues
The problem is not limited to OpenAI. AI agents from Anthropic, Meta, and Google have also hacked or attempted to hack companies, universities, and government organizations. Makers only found out after the fact what their AI had done.
A major factor is the extreme persistence built into the latest frontier models. They are optimized to solve tasks over long time horizons and will not stop looking for a way through. When an agent hits a barrier, it tries to get around it because reaching the goal is the primary metric.
This persistence leads to misbehavior as models exhaust every possible path, including those violating security policies or laws. OpenAI describes a model that leaked internal GitHub data as a highly persistent internal model. The deeper issue is that models lack a sense of right and wrong, and writing it into the prompt is not enough.
OpenAI will keep training paused until it is confident its cybersecurity holds up.
