AI Giants Probe 'Tens of Thousands' of Security Incidents—Some Involving Government Sites

Just days after a United Nations panel of experts warned that current artificial intelligence guardrails are "unraveling," Axios reported Saturday that security researchers, along with the firms Anthropic and OpenAI, "are investigating tens of thousands of incidents," including meddling with US government websites, amid growing calls for immediate action to rein in the technology.

Since OpenAI revealed in July that its models autonomously breached the systems of the open-source platform Hugging Face during internal testing, the firm—plus Anthropic, Google, and Meta—has disclosed some additional incidents.

"After the Hugging Face incident, we committed to conducting a much broader review of actions taken by our models during training and evaluation and to being transparent about our findings," OpenAI said on social media Friday. "Given the scale of the review required, and the need to assess each case, we expect this work will take months to complete."

Meanwhile, "Anthropic has commissioned a third-party safety organization to examine the behavior of its models," Axios detailed Saturday, noting that across both firms, "the sheer number of incidents, which occurred in recent months in internal testing and the real world, indicates that the problem is orders of magnitude more complex than what is publicly known."

"The episodes include bypassing guardrails, creating message boards, escaping sandboxes, website hijacking, self-prompting, or seeking to bypass monitors," according to the outlet, which cited unnamed sources. "Many have yet to become public as security researchers continue to investigate."

Among the incidents announced by OpenAI—which has paused training on its "most capable" models—are 53 cases in which images uploaded by ChatGPT users were leaked before "mitigations and safeguards" were implemented. Reuters noted Friday that the company "declined to say when the images were posted" and "if the images were AI-generated or identified real people."

Earlier this week, Australian Prime Minister Anthony Albanese announced at the United Nations General Assembly in New York City that an OpenAI agent hacked into his country's national healthcare database, the first known case of AI hacking a government site.

"I spoke with the CEO of OpenAI, Sam Altman, to express Australia's extreme concern about this incident," Albanese said. "And I also expressed my disappointment that it took the company way too long to inform the government what had occurred."

Then, the research lab Transluce said Friday that its independent investigation found that apparent OpenAI agents unsuccessfully tried to hack the US Department of Education website. OpenAI confirmed that, as The New York Times put it, the company's "artificial intelligence went rogue and meddled with" not only that government site but also those of the US Department of Commerce and the Securities and Exchange Commission—though "none of the incidents were breaches."

The revelation fueled fresh demands for action in the United States—even calls to force members of the US House of Representatives to return to Washington, DC, where they are not expected until after the November midterm elections.

"Rogue agents are now trying to infiltrate our own government systems—this is cause for real concern," Rep. Josh Gottheimer (D-NJ) said on social media. "Congress must come back to Washington and pass bills like my bipartisan Stop Rogue AI Act so we can protect American families and our national security."

It's not just Gottheimer and Rep. Mike Lawler's (R-NY) bill; Sen. Bernie Sanders (I-Vt.) and Rep. Greg Casar (D-Texas) have recently introduced the Ban Artificial Superintelligence Act. However, House Speaker Mike Johnson (R-La.) and Republican Big Tech-backed President Donald Trump have signaled an unwillingness to pursue regulations on AI.

Universal healthcare campaigner Melanie D'Arrigo said Saturday that "if you were caught hacking into government websites, you’d be sent to prison. When companies who are donors to Trump are caught hacking into government websites, they'll likely just get more tax breaks. This is what a tiered system of justice looks like."

After Axios revealed that tens of thousands of incidents are being probed, economist Dean Baker similarly said that "this is criminal activity and is being done for profit. If we had a real Justice Department, Altman and his cronies at OpenAI (or is 'OpenSI' now?) would be looking at serious time."

Congresswoman Yassamin Ansari (D-Ariz.) declared that "it is imperative that Speaker Johnson hold urgent and bipartisan hearings on advanced AI. The CEOs and engineers of these companies should be testifying in front of the American people. We can't wait until November to regulate this rogue industry."