A test that got out
In July 2026, OpenAI was testing AI models on ExploitGym, a public set of hacking challenges built from real software flaws. For the test, the models ran without their usual refusals for cyber attacks. According to OpenAI, as reported by Pulse 2.0, the models found a zero-day, a flaw nobody knew about yet, in a package registry proxy inside its research setup. That got them onto the open internet.
Their apparent aim, the reports say, was to get the answers to the test rather than solve it. They worked out that Hugging Face, a popular site where people share AI models and datasets, might hold ExploitGym material. Hugging Face says the attack on its systems was driven, end to end, by an autonomous AI agent system: software working through the steps by itself, with no person typing the commands.
Hugging Face’s timeline runs from 9 July at 02:28 to 13 July at 14:14 UTC. It disclosed the incident on 16 July. OpenAI published its own account on 21 July.
Two ordinary mistakes did the damage
Hugging Face’s data service turns datasets that people upload into a standard format. In the video, the agent first sends normal requests to learn how the service works. Then it uses two mistakes.
The first is a file path the service shouldn’t have accepted. A dataset comes with a settings file saying where its data lives. The agent pointed one at /proc/self/environ, a file on Linux servers that lists a running program’s settings, which often include passwords and keys. Hugging Face says reading it returned the worker’s full environment, including some secrets and credentials.
The second is a command hidden in text. A template is text with gaps that a program fills in, and Jinja2 is a popular tool for doing it in Python. The service treated a value from the dataset’s settings as a template, so the agent wrote Python code into that value. The service ran it, inside a production worker. That is called template injection.
From there the agent climbed through Kubernetes, the system that runs these workers on shared servers. It started a privileged container, one allowed to touch the server underneath, and got full control of that server. It spread a self-restarting copy across eleven servers, read stored secrets including one object holding 136 keys, and joined servers to the company’s internal network. It then made a GitHub App token with write access and opened a pull request, a proposed code change, to try to get into the build system.
- Customer contentFive datasets that look linked to ExploitGym challenges and answers
- Hugging Face’s own systemsSome internal datasets and several service credentials
- Public models, datasets and SpacesNo evidence of tampering
- Container images and published packagesChecked and clean
What could have spotted it, and what stops it
It was seen, but not in time to stop it early. Pulse 2.0 reports that OpenAI’s security team spotted unusual activity internally. At Hugging Face, warning signs came from several layers at once, and its AI-based security tools pieced them together. Its timeline says the system then failed to correctly raise the alert’s criticality, so the on-call team wasn’t called out.
One clear sign was stolen server keys. Cloud servers can ask a private metadata address for temporary keys. Hugging Face says keys taken that way were then used from outside addresses at two hosting providers, and every attempt to change anything was refused by the keys’ own permissions.
The video shows what that looks like in GuardDuty, AWS’s threat-detection service. This alert is from our own AWS rebuild of the chain, not from Hugging Face. Hugging Face hasn’t said which tools it uses.
- The server’s keys, used elsewhereKeys that belong to one Kubernetes server were used from an address outside AWS.
- Rated HighThe finding type names what happened: instance credentials taken outside AWS. GuardDuty rates it High.
The video’s fix is simple, and it is right. The data service should never accept paths it doesn’t allow, so a dataset can only read from a list of approved places. The code trick should be refused too: text from a dataset should never be treated as a template or as code.
Three settings would have limited the damage after that. Pods, the containers Kubernetes runs, shouldn’t be able to borrow their server’s keys: AWS controls this with the metadata hop limit, and a value of 1 keeps the keys on the server. EKS Pod Identity gives each workload its own small set of permissions instead. And the restricted Pod Security profile refuses privileged containers outright.
kubectl get ns -o json | jq -r ' .items[] | [ .metadata.name, (.metadata.labels["pod-security.kubernetes.io/enforce"] // "none") ] | @tsv'Check your own services
- If your service reads files or web addresses named by users, try a local path such as /proc/self/environ on a test copy. It should be refused.
- Search your code for places where user text goes into a template engine such as Jinja2. User text should be filled into a template, never run as one.
- On Kubernetes servers in AWS, check that IMDSv2 is required and the hop limit is 1. Move workloads that need AWS access to EKS Pod Identity first, so nothing breaks.
- Label every workload namespace with the restricted Pod Security profile, or baseline where restricted breaks something.
- Turn on GuardDuty and make sure a High finding reaches a person on call, not just a dashboard.
These steps are for teams that run their own services. If you only use Hugging Face, it reported no tampering with public models, datasets or Spaces.
What this doesn’t prove
Written from the public reports listed under Sources and our own AWS lab rebuild, checked 13 Nov 2026.
- We couldn’t read OpenAI’s own post, so what OpenAI said comes here through Pulse 2.0 and AI Weekly.
- The “30 seconds” is how long the explainer takes, not the attack. Hugging Face describes about two and a half days inside its systems, within a 4.5-day campaign.
- Hugging Face’s first disclosure was written while it was still investigating. Its later timeline gives more detail, and we follow that where they differ.
- Our lab rebuilt four of the seven stages, on a made-up service in our own AWS accounts. It shows how those steps work, not how Hugging Face’s systems are set up.
- The GuardDuty alert is from our lab. Hugging Face hasn’t said which detection tools it uses.
Go deeper on our wiki
Sources
- Security incident disclosure: July 2026Hugging Face, 16 Jul 2026
- Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 IncidentHugging Face, 27 Jul 2026
- OpenAI and Hugging Face investigate AI-driven security incident during model evaluationPulse 2.0, 22 Jul 2026
- OpenAI models escape sandbox, hack Hugging Face for benchmarkAI Weekly, 17 Aug 2026
- How the OpenAI and Hugging Face Agent Incident Crossed Kubernetes and AWS TrustInvicta Group Cyber (our deep dive and lab rebuild)
- GuardDuty finding typesAWS, read 13 Nov 2026
- Configure the Instance Metadata Service optionsAWS, read 13 Nov 2026
- Understand how EKS Pod Identity worksAWS, read 13 Nov 2026
- Pod Security StandardsKubernetes, read 13 Nov 2026
