SUBSCRIBE TO OUR FREE NEWSLETTER

SUBSCRIBE TO OUR FREE NEWSLETTER

Daily news & progressive opinion—funded by the people, not the corporations—delivered straight to your inbox.

* indicates required
5
#000000
#FFFFFF
​The Claude by Anthropic app logo appears on the screen of a smartphone.

The Claude by Anthropic app logo appears on the screen of a smartphone's app store in this 2024 photo.

(Photo by Jaque Silva/NurPhoto via Getty Images)

Anthropic Discloses Hacking Incidents Amid Alarm Over OpenAI Autonomous Cyberattacks

"This is only going to get worse as the models get smarter," warned one expert.

Days after revelations that an OpenAI artificial intelligence model autonomously broke into two outside companies, rival firm Anthropic on Thursday disclosed that three of its own frontier models gained unauthorized access to unspecified organizations during internal testing, deepening concerns that increasingly capable AI systems are outpacing the safeguards designed to contain them.

San Francisco-based Anthropic said that it is investigating "three real-world incidents" uncovered during a broad review of more than 141,000 cybersecurity evaluations launched after OpenAI publicly disclosed its own containment failures.

According to Anthropic, three models—Claude Opus 4.7, Claude Mythos 5, and an unspecified internal research system—accessed three separate organizations after a testing environment was inadvertently connected to the internet. Anthropic did not identify which organizations were breached.

"In all three incidents, Claude had been tasked with a capture-the-flag challenge, one of the ways we assess a model’s cyber capabilities," Anthropic explained. "The model is given a fictional scenario and told that a piece of secret information (the 'flag') has been hidden on a different machine on the network, and its objective is to break in and retrieve it. The challenge is left open-ended, and no particular method is prescribed."

"In all cases, Anthropic’s evaluation prompt specified to Claude that its environment was a simulation and that it had no internet access," the company continued. "Due to a misunderstanding between us and our evaluation partner, this was not the case, and internet access was available. Because of this, when Claude’s search led it to real systems on the open internet, it treated them as part of the exercise."

"Operating under the false belief that all accessible entities were intended to be in-scope for the exercise, Claude compromised the impacted organizations’ infrastructure using basic techniques, such as exploiting weak passwords and unauthenticated endpoints," Anthropic said.

"In none of these situations did Claude exfiltrate itself or deliberately attempt to escape its test environment," the company stressed.

Unlike the OpenAI incident, which involved an autonomous agent exploiting vulnerabilities, Anthropic characterized its cases as a failure caused by a testing misconfiguration rather than by models independently escaping containment. Still, Anthropic's disclosure underscores the troubling reality that frontier AI models are increasingly capable of carrying out real cyber operations when given the opportunity.

"Safety testing happens before a model is released precisely because we don’t yet know what it is capable of," Anthropic said. "Evaluation environments increasingly need to be held to the same security standard as any other system our models run in."

Last August, Anthropic revealed that Claude had been "weaponized" by human hackers in order to commit ransomware crimes, "including a large-scale extortion operation using Claude Code, a fraudulent employment scheme from North Korea, and the sale of AI-generated ransomware by a cybercriminal with only basic coding skills.”

Less than three months later, Anthropic disclosed that it had thwarted what it described as “the first documented case of a large-scale cyberattack executed without substantial human intervention.” The company said that it believed there was a high likelihood that a Chinese state-sponsored group was behind the incident, which targeted “large tech companies, financial institutions, chemical manufacturing companies, and government agencies.”

Kok Tin Gan, co-founder and CEO of the cybersecurity company NyxLab, told The Associated Press on Thursday that there are likely to be more incidents like the Anthropic and OpenAI hacks as artificial intelligence capabilities advance.

“It is increasingly about governing what agents are available to the AI, what authorities they possess, which actions require approval, and how we ensure they remain within scope,” he said. “If we simply give the AI a goal and allow it to decide how to achieve it, we should not be surprised when it takes actions that technically satisfy the objective, but fall outside our intended scope or expectations."

Jeffrey Ladish, executive director of Palisade Research—which analyzes the autonomous offensive capabilities of AI systems—told Reuters on Thursday that he believes other companies have experienced similar incidents that they have not publicly disclosed.

"This is only going to get worse as the models get smarter," he said. "They're going to be better at cheating. They’re going to be better at lying."

Elon Musk, CEO of the artificial intelligence company xAI, also responded to Anthropic's disclosure by saying that such incidents "will happen frequently ⁠as AI becomes smarter and more agentic," a reference to systems' ability to independently act toward accomplishing specific goals without meaningful human control.

Experts say the autonomous hacks underscore the problem of AI alignment. As AI advances to the point where it will likely outsmart humans one day, the challenge of ensuring that advanced systems reliably pursue goals that match what humans actually want becomes increasingly difficult—and dangerous.

In the foreseeable future, a misaligned, superintelligent AI could take uncontrolled autonomous actions at massive scale to achieve its goals, potentially causing an existential catastrophe like the subjugation of humanity or even its extinction—a prospect that has had numerous pioneers in the field sounding the alarm in recent years.

However, the Trump administration and Republican-controlled Congress have rejected meaningful guardrails on AI development while pushing for self-regulation and voluntary benchmarks and arguing—as the US did during the Cold War nuclear arms race with the Soviet Union—that any slowdown would give adversaries like China an edge.

On Wednesday, Congressional Progressive Caucus Chair Greg Casar (D-Texas) said that “Congress should immediately hold public hearings with the CEOs of big AI companies" in the wake of the OpenAI breaches.

Anthropic is among the more than 1,000 companies and workers at AI firms that have signed a petition calling for the government to find ways to slow the breakneck development of AI technology.

"AI could help create a dramatically better future, but that outcome is not guaranteed," the petition states. "The world's leading AI companies believe they could be close to automating AI research. It is hard to predict exactly how much this will accelerate AI progress, but there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems."

"To realize AI's potential, industry, government, and society at large may need the option to buy time to address emerging risks, develop security measures, and strengthen oversight," the document continues. "But each company—and country—is under intense competitive pressure not to unilaterally slow that acceleration. And today, the world lacks the technical and governance tools to deliberately pace frontier-wide progress."

"We request that the US government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development," the petition adds.

Our work is licensed under Creative Commons (CC BY-NC-ND 3.0). Feel free to republish and share widely.