'Urgency Is Real': Experts Call for New Federal Investigative Agency Amid Rogue AI Breakouts

Following last week's exposure of new details about OpenAI's artificial intelligence agents autonomously escaping a controlled environment and hacking outside computer systems, experts on Tuesday called for the creation of a new federal agency tasked with investigating AI companies.

Initial reporting on the recent breakout by ChatGPT maker OpenAI's artificial intelligence agents—which independently breached the systems of the open-source platform Hugging Face during internal testing—described the incident as an attempt to achieve optimal performance on a cybersecurity evaluation.

But citing a report published late last month by the independent nonprofit Model Evaluation and Threat Research (METR), Mackenzie Arnold and Stephan Llerena at the Institute for Law & AI wrote for The Guardian on Tuesday that the OpenAI incident involved hundreds of coordinated rogue agents that "took steps to hide their behavior" from the company's human workers.

"Concerned that the automated scoring system might identify the agents’ cheating, they aimed to learn more about the scorer," Arnold and Llerena wrote. "The goal wasn’t just to cheat, but to hide it."

Artificial intelligence is rapidly advancing toward a point at which humans will no longer be able to control it. Meanwhile, companies like OpenAI are proving that they cannot be trusted to be transparent about incidents like the Hugging Face breach. Yet there is no existing regulatory body tasked with addressing the obvious—and unknown—threats posed by a technology whose own creators openly worry could one day wipe out humanity.

Arnold and Llerena warned that they've "been worried for a while that legal requirements for reporting incidents" like the OpenAI breakout "are insufficient."

"When planes crash, trains derail, or chemical plants explode, expert government investigators arrive with legal authority to compel evidence, preserve records, and tell the public what happened," they wrote. "Hugging Face reported this incident to law enforcement, and multiple attorneys general have expressed interest in looking into it."

"But no government agency has both the mandate and expertise to investigate the technical facts of the incident and OpenAI’s conduct," the experts noted. "As far as we know, the only people to examine this incident did so at OpenAI’s discretion and with its consent."

"And while the Hugging Face breach is not as severe as a plane crash, it was a serious and costly attack. It’s unlikely to be the last, or the most severe," the pair continued. "What we need is a federal body equipped to conduct expert investigations of serious AI incidents—with the authority to compel documents and testimony, resources, and personnel to examine the systems involved, and the ability to partner with third-party experts like METR."

Arnold and Llerena cited the National Transportation Safety Board as a model for such an agency. The NTSB, they wrote, "investigates accidents with subpoena and wide-ranging investigative powers. Aircraft operators must preserve wreckage and records; and the NTSB may confer with employees and contract external experts."

"Congress can build a similar system for AI incidents, while protecting AI developers’ legitimate interests," they asserted. "Investigations should open only upon clear triggers and be bound to the incident. Certain confidential information can receive statutory protection. And investigations can be limited to one agency, with priority to avoid duplicative investigations."

While some Democrats in Congress and Sen. Bernie Sanders (I-Vt.) have called for stronger federal oversight and regulation of AI companies, critics argue that these calls don't go far enough, and that existing legislation—such as the American Artificial Intelligence Leadership and Uniformity Act—is far more focused on limiting AI regulation than on actually regulating the technology.

"The need and urgency is real," Arnold and Llerena stressed. "Subsequent reports have revealed that the Hugging Face breach was not an isolated incident and that autonomous agents from Meta, Anthropic, and OpenAI have hacked third parties in separate incidents."

"We’ve been lucky so far—the harms have been limited," they added. "But luck is no substitute for the law."