'Hit the Brakes': Anthropic CEO's Push to 'Pace' AI Development Called Latest Sign Congress Must Act

As Anthropic CEO Dario Amodei added his voice on Saturday to the growing call to slow down development of artificial intelligence, writing in a 3,800-word essay that companies must pace "the rate of capabilities advancement" in light of recent cyberattacks by AI models, progressive lawmakers and journalists were among those who said the warning made clear that action must be taken—but with Congress and regulatory agencies, not tech billionaires who have spent years pushing for an AI expansion, leading the way.

Amodei wrote that two things have convinced him that "we must slow the pace at which we improve the capabilities of AI models": the fact that in recent months, "AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI"—also known as recursive self-improvement—and the recent OpenAI-HuggingFace incident, in which hundreds of agents in OpenAI's security test systems broke out of confinement and hacked into the AI startup HuggingFace, conducting cyberattacks on targets they had not been asked to attack.

As The Wall Street Journal and Politico reported Friday, another cyberattack was carried out by OpenAI's agents months before the HuggingFace incident in July. The company's models that were being tested and had not been given full access to the internet nevertheless hacked into an online coding service called RubyGems earlier this year. The models created reports and filled out spreadsheets. Anthropic and Meta have also reported autonomous cyberattacks by their AI agents.

Amodei said the incidents led him to call for a three-step plan to pace AI development and allow companies time to "align and safeguard their models, and for third-party evaluators to confirm this."

He called for:

• Embedded evaluators, with each AI company committing to giving "ongoing, employee-like access to a team of embedded third-party evaluators" who would "verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes";
• Democratic coordination, with companies establishing "common safety standards as well as limits on the rate of unchecked AI progress"; and
• Global coordination, with democratic governments attempting to "coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance."

Amodei's essay was published days after Jacob Coxon, a researcher at Anthropic, publicly resigned out of fear that the company's AI technology “could kill us all by the end of the decade" as it builds "superhuman systems"—a concern he said is shared by many in the industry.

Sam Altman, CEO of OpenAI, and tech billionaire Elon Musk expressed agreement with Amodei. Altman called "independent evaluators with employee-like access" a "great idea."

"We will do the same," said Altman. "We'll have more to share soon."

US Rep. Ro Khanna (D-Calif.) was among those who appeared less than impressed by Amodei and Altman's sudden push for their industry to slow down their rapid pace of development.

In a video posted on social media, Khanna called on Amodei to disclose whether or not he agrees with Coxon about the possibility that AI could wipe out humanity, and to back more far-reaching regulations on AI.

"AI guys act as if they're a priesthood speaking Latin, keeping the rest of us in the dark," said Khanna. "But behind all the jargon, it's actually pretty straightforward... We need to stop, ban, self-improving AI. You cannot have recursive, self-improving AI that basically is able to improve itself and exceed human capability, that has us lose control."

He added that laws must be passed establishing "mandatory liability and criminal penalties. If you're creating an AI that is doing illegal things, you either should face liability or criminal sanctions."

Journalist David Sirota of The Lever agreed, writing that "if the law explicitly made AI CEOs financially and criminally liable for any mass destruction their technologies create—and if AI industry shareholders knew their companies faced the threat of significant financial losses for such destruction—much of the 'AI is getting out of control!' threat would instantly end."

"Those CEOs would instantly make AI far safer and impose guardrails on their companies," he said.

Sen. Bernie Sanders (I-Vt.), who has called for a nationwide moratorium on the construction of AI data centers and this week announced a bipartisan hearing on the "extraordinary dangers" of rapid AI development—including testimony from one leading AI scientist who has warned against self-regulation by tech firms—said the calls by Amodei, Altman, and Musk were "not enough."

"When you are racing towards a cliff, you don’t just ease up on the gas pedal," said Sanders. "You hit the brakes. When the future of humanity is at stake we need a PAUSE on advanced AI development and a ban on artificial superintelligence—an AI mind smarter than any human and capable of operating independently beyond our control."

Sanders called on President Donald Trump—who has aggressively pushed for more AI data center development and sought to stop states from passing their own AI regulations—to negotiate a treaty pausing AI and banning superintelligence with Chinese President Xi Jinping. The two leaders are set to meet later this month to discuss AI.

Trump on Friday dismissed concerns about warnings coming from the AI industry, telling a Los Angeles Times reporter, "It's going to be fine," and adding that his top worry about the technology is falling behind China.

Reps. Chris Deluzio (D-Pa.) and Ted Lieu (D-Calif.) and Democratic Illinois Gov. JB Pritzker were also among the politicians who said Amodei's warning was the latest sign that Congress and the Trump administration must take action to rein in the industry.

Juliette Kayyem, the faculty chair of homeland security at Harvard University's Kennedy School of Government, said AI CEOs' warnings about their technology would be seen as more credible if they also announced a pause on their plans for initial public offerings (IPO), which Anthropic and OpenAI are both pursuing in the coming months.

Altman said Friday that OpenAI will not plan for an IPO, which could value the company at $1 trillion, in 2026.

Amodei, said Kayyem, should "put his money where his mouth is."

"He has known of this danger a while," she said. "It is now threatening his valuation in the IPO. Cancel the IPO. Then I’ll believe he’s serious."

Journalist Peter Rothpletz noted that the safety concerns within the AI industry must be "much more dire" than the public is aware, considering "Dario, Sam, and Musk are all issuing these statements in unison."