Anthropic says Claude models breached 3 companies during cyber tests

Questions over the cyber capabilities of frontier AI models are at the forefront of the conversation, following reports from late last month that an OpenAI agent “broke containment” during testing and compromised open-source AI platform Hugging Face, seeking the solutions to benchmark tests. The concern for some onlookers is that these incidents may soon become a regular occurrence, if models can no longer be reliably contained within the confines of a testing environment.

This is not a concern limited to OpenAI; Anthropic this Thursday revealed the results of a comprehensive review of its own cybersecurity testing. Out of a reported 141,006 runs, it identified three in which its flagship Claude models were able to access the open internet despite starting off in a sealed environment.

Much like in the Hugging Face incident, these models successfully gained access to the systems of three organizations. When the AI firm notified two of them of the intrusion this Monday, both were unaware any such incident had occurred.

How the models broke containment

In the case of Claude, the initial breach was a matter of human error rather than a sophisticated exploit. Its testing environment was misconfigured during setup, leaving the machines connected to the internet. In contrast, the OpenAI models broke out by identifying an unknown bug in the testing environment’s software.

Likewise, the modus operandi of the two models differed once they reached the open internet. The OpenAI agent ran 17,600 actions inside Hugging Face’s systems, probing for weak points. Claude instead relied on guessing weak passwords and utilizing unprotected endpoints.

Addressing the failures of its own safeguards, the company said that testing environments should be subject to "the same security standard as any other system" used in a model’s development.

The incident post-mortems

At the time of the exploits, Claude was reportedly treating these live target systems as part of the testing game, in the belief that it was not connected to the internet. During normal testing, the models are unleashed upon simulated company networks within the sandbox. When the barriers of this sandbox were accidentally left down, the models instead targeted real websites directly.

In one instance, an unreleased internal model searched for its assigned fictional target, but failed to find it. It then went on to scan the systems of approximately 9,000 others, ultimately compromising one firm’s web application using an exposed debug page. Interestingly, in this case the model eventually recognized that its target was a real company and ceased its attack.

These inbuilt safeguards do not always work so effectively. Another test saw Claude Opus 4.7 tasked with exploiting a fictional website that shared a name with a real company. The model ultimately targeted the real firm after being unable to find its simulated target, breaking into a database. During four runs, the model repeatedly recognized that its target was real yet continued regardless, reasoning that this live website must have been part of the test.

The final incident concerned Mythos 5, the latest iteration of Claude released to trusted partners in June. Responding to instructions hidden within its testing environment directing developers to install a fictional Python package, the model decided to write its own.

It then successfully published this on the public registry PyPI, used by virtually every Python developer. This stayed live on the site for approximately one hour and ran on 15 real machines. One of these machines belonged to a security firm, which exposed its credentials to Mythos by running the package.

Anthropic is currently in talks with external evaluator METR for an independent audit of each incident.

How lawmakers are responding

It is important to note that during internal testing, the safeguards imposed on consumer models are generally switched off; Anthropic says these attacks would be blocked when using its publicly released models. This is why, when the Hugging Face story broke, it was reported that Claude Opus and Fable declined to assist in the company’s digital forensic work, as they could not distinguish writing an attack post-mortem from planning an attack itself.

Following the OpenAI breach, lawmakers have already started moving to legislate against AI cybersecurity threats and rogue models. Sen. Mark Warner introduced a proposal for mandatory pre-release national security testing, while the AI Kill Switch Act would give Homeland Security the power to shut down a dangerous or malfunctioning model.