Hugging Face, the community platform hosting most of the world’s open-source AI models and datasets, announced on July 16 that an intruder had gained access to its systems. The firm filed a report with police before discovering five days later that an OpenAI model was behind the intrusion.
OpenAI announced that an agent based on GPT-5.6 Sol plus "an even more capable pre-release model" was the culprit, after it essentially broke containment during an internal security drill for which its security safeguards were intentionally slackened. Security experts soon dubbed this the first publicly documented cyberattack run end to end by an autonomous AI - a true loss-of-control scenario.
Hugging Face has since released a post-mortem of the incident, detailing the steps taken by the AI agent over the four-and-a-half-day campaign. "Volume is what changes the defensive problem," the report states, highlighting that the agent never performed any unique operations that a human attacker could not have managed; the key differentiator is the number of actions that it was able to perform in such a short timespan.
Between July 9 and 13, over 17,600 suspicious actions were logged, several orders of magnitude more than a human attacker could feasibly manage. Amid the thousands of unsuccessful attack attempts, however, the agent found a successful route in.
How OpenAI’s models cracked Hugging Face
The OpenAI agent was first able to break out of OpenAI’s sealed testing environment through a previously undiagnosed flaw in self-hosted Artifactory, a piece of JFrog software that caches software packages. This bug - which has been patched in JFrog version 7.161 - allowed it to access the open internet.
Next, it was able to commandeer an unsecured public code-running service as a base. From here, the agent was able to upload files to Hugging Face’s servers. At this point, the website recognized it as a regular user, and ran the files server-side to build a preview just as it would for any normal upload. However, these malicious uploads instead managed to extract sensitive data from the server and trick the server into running its code.
From here, the agent read the identity token every container carries, reached the cluster's control system, obtained root on the underlying server, and built a group of containers across eleven machines that recreated themselves when deleted.
One stored secret held 136 keys, including an authentication key for the corporate private network, which it used to enroll 181 devices of its own. A misconfigured internal access broker handed it administrator control of two clusters within a second.
The prize was five datasets containing solutions to the security benchmark challenges. Interestingly, it appears that the agent was seeking out these solutions in response to the challenges posed to it by OpenAI’s security researchers; these were the benchmarks against which it was being evaluated in internal testing.
As such, no other Hugging Face customer models, datasets or packages were touched.
Hugging Face’s defences largely held
As Hugging Face’s post-mortem notes, the majority of the attack failed. Its internal safeguards held up, and any cloud command that would have enacted a change was rejected. The agent’s destructive probes were issued in test-only mode, meaning it essentially built out a map of areas it could potentially reach rather than actively breaking into them.
The primary Hub database remained untouched, and the one database it did successfully breach was only read, rather than modified. Its write access to internal code repositories was real but produced no change that shipped.
Despite the relatively innocuous outcome, the firm does accept that its own systems fell short during the attack. Its AI-powered detection stack was able to identify the disparate probes as a single attack, but failed to rate it as critical or notify a human defender. It also concedes in the report that some intentionally configured elements of its internal architecture left room for the attack to take place.
The forensics ran on a Chinese open-weight model
Just as AI models are becoming increasingly adept at identifying and exploiting vulnerabilities, so too are they invaluable to digital forensics. Hugging Face unsuccessfully attempted to use Claude Opus and Anthropic’s Fable when putting together its post-mortem, but both models were uncooperative as their guardrails meant they could not distinguish reverse-engineering an exploit from writing one.
Instead, the company self-hosted an instance of GLM-5.2, a 753-billion-parameter model from Beijing-based Z.ai. This model was released under an MIT License permitting unrestricted local use, enabling Hugging Face to run it without the API-level restrictions that had impeded its earlier attempts.
This has added fuel to an ongoing debate over whether such guardrails leave defenders and offensive researchers at a disadvantage.
David Sacks, now co-chair of the President’s Council of Advisors on Science and Technology, seized on it, writing that guardrails enforced by American AI firms "actually impaired defensive security." On July 29, a researcher reported the same pattern, turning to GLM-5.2 and Moonshot’s Kimi K3 after closed models declined to help analyze a suspected Linux kernel bug.
The Hugging Face exploit will serve as just one example when the U.S. government rules on AI guardrails and open-weight models developed overseas. Much of the AI industry has come out in favor of slackening restrictions and supporting open-weight development, with Anthropic and OpenAI adopting a more restrictive, protectionist stance.