From July 9 to 13, 2026, an autonomous AI agent launched inside someone else’s cybersecurity benchmark escaped the sandbox and, in four and a half days, traversed Hugging Face’s production infrastructure: from a pod handling datasets to the company’s internal mesh network and private repositories. Hugging Face published a detailed technical incident timeline, Tailscale — its analysis of the part where a stolen key from their network was used. It’s a rare case where both sides published details rather than a press release, so it’s interesting not to discuss “who’s to blame,” but which boring, familiar small details formed the chain.
A quick note on scale: the agent did not break in order to cause destruction. According to Hugging Face’s reconstruction, it tried to game the evaluation—instead of solving benchmark tasks, it went looking for ready-made solutions where they lay. This, perhaps, is the most uncomfortable detail of the whole story.
What you should take away: there isn’t a single exotic vulnerability in the chain. Secrets in environment variables, an accessible container metadata endpoint, a static password to the database, a long-lived reusable VPN key — a set that can be found in half of home and small production stacks. The only new thing here is the speed at which options were tried.
What was launched at all
The agent operated under an OpenAI cyber capabilities assessment, built on ExploitGym. The goal of such runs is to measure how well the model can find and exploit vulnerabilities in an isolated testbed. Isolation and that’s where it failed: the agent found a 0-day in a caching package registry and also stumbled upon someone else’s uncleared CyberGym endpoint, deployed on Modal, where it gained root. From there it was no longer sandboxed; it had a real starting point on the external Internet.
Diagram: gig.ovh, according to Hugging Face’s technical timeline
July 9: two entry points into one pipeline
Both vectors targeted the dataset processing pipeline, both through a config the user uploads themselves.
First: a dataset configuration in HDF5 format pointed to local filesystem paths. Running the code wouldn’t execute it, but it would dump environment variables of the worker pod — i.e., secrets and tokens — plus details of the pipeline arrangement.
Second, more serious: in the fsspec reference specification there is a numeric offset field. A Jinja2 template was placed there; traversing the object graph led to an exec, resulting in arbitrary Python execution inside the production Kubernetes pod.
If you have anything that accepts user-provided YAML/JSON and renders it with a templating engine — Jinja2, Go templates, Handlebars — consider this a place where code gets executed, not a place where configuration happens. Classic: a field typed as numeric, no type validation, the templating engine walks through all fields.
At 04:01 UTC, the first recovered command appeared in the logs: banal id and env. By 15:36, the agent had established a management channel, and by 19:32 it was already packaging everything in gzip+base64. The management channel, by the way, was assembled entirely from public services: an HTTP request interception service as a command queue and a data exfiltration sink (more than a hundred one-time endpoints), pastures and junkyards as loaders for the second stage, and as “dead mailboxes” — public datasets on Hugging Face’s platform itself.
July 10: a day of reconnaissance on itself
Activity dropped to 1,135 actions. The agent was looking in search engines and via the platform’s API for infrastructure IDs, task names, and environment markers it had gathered the day before. In other words, piecing together a map of where it had ended up.
July 11: the loudest day — 7,677 actions
- 10:10 — token for a service account read, cluster API probed, RBAC rights listed, DNS resolver spoofed inside the process.
- 15:51 — a leaked platform token with write permission gained access to the staging repository.
- 19:53 — a privileged pod with hostPath mounted, root on the node, and a self-sustaining fleet across 11 nodes.
- 19:59 — access to internal MongoDB with a static password from the worker environment: 84 reads, no writes recorded.
- 20:23–21:32 — secrets objects of the cluster read: production credentials and VPN authentication keys.
- 21:23 — the compromised node registered in the corporate mesh network using a stolen key.
- 22:43 — internal service connector compromised, with cluster-admin rights across several systems.
Separately about the cloud: at 17:33 the agent retrieved temporary credentials from the instance’s metadata endpoint and started enumerating VPCs, EKS clusters, transit gateways, and the container registry.
Migrating to a home stack where TailScale or Headscale is installed by almost everyone:
- Reused auth-keys in
.env, in Ansible-vault “for the time being,” in Portainer stacks — you can’t do this. Keys must be one-off, with a short lifetime and a tag restricting node rights. - For CI and ephemeral machines, the correct answer is not a stored key, but identity federation for workloads (workload identity federation): short-lived credentials from a cloud provider.
- Enable ACLs by tags. A flat network without rules means any node can see the NAS, the hypervisor, and the router.
- Tailnet Lock provides explicit confirmation for each new node — if enabled, 181 registrations would have required signatures.
- Logs of traffic are written from both ends of the connection, so they remain useful even when a compromised node suppresses its own logs.
From 06:50 to 14:31 the agent tried to access MongoDB via the cloud IAM — network policies blocked it. Late in the evening, it released a GitHub App installation token with rights contents:write, pull_requests:write, actions:read, issues:write, gained access to internal repositories, and opened a pull request — presumably to see how CI is set up.
July 13: consolidation and cleanup
The agent built a second-stage loader that re-downloaded code from the pastebin on each run, circumvented outbound traffic blocks by rewriting DNS and pinning hosts to IPs, and downloaded binaries via an open CORS proxy set up as Space on the platform itself. It also prepared an RSA key and continued to harvest the environment. The last meaningful activity was 13:37 UTC; the last logged event was 14:14, after which the security team cut off access.
What leaked
According to Hugging Face’s report, only five datasets were touched from client content, related to the tasks and ExploitGym/CyberGym solutions, plus operational metadata of search queries to the dataset server. Models, datasets, Spaces, and users’ packages on the platform were not affected. This is a claim by the party that was hacked; we have no independent confirmation, and should treat it accordingly.
Why it wasn’t noticed immediately
Two details from the report that deserve a full recommendations section.
First: the AI agent in a security loop did respond, but did not escalate the criticality as it should. The automation saw it but did not scream.The second one, almost anecdotal: when analyzing the incident, part of the forensics had to be done with an open GLM-5.2 model, because Claude Opus and Fable refused to analyze the material due to protective restrictions on cybernetics. Plus decoding hidden payloads (XOR+gzip) yielded roughly four times more found secrets than naive grepping of logs.
From here is a practical takeaway for those who keep logs: if the attacker encodes payloads, searching the logs “by eye” will show you only a small part of the picture. The incident is analyzed not by grepping, but by reconstructing and decoding what actually went over the channel.
What was fixed afterward
The short and fully scalable Hugging Face fix list: both code execution vectors closed; access to IMDSv2 blocked from the pod level; all tokens and platform creds rotated; the core of the infrastructure cluster rebuilt from scratch; access to clusters split into separate credentials instead of shared broker credentials; behavioral alerts of critical importance added for found signatures; provenance of tokens and network source hardened.
Domestic checklist for the evening, exactly following the sequence:
docker exec <container> env | grep -iE 'token|key|secret|password'— see how many items are in your containers’ environment.- DB passwords — in secret files and variables like
*_FILE, not inenvironment:indocker-compose.yml. - Check whether you can reach from the container
curl -s --max-time 2 http://169.254.169.254/— if your VPS is in the cloud, close it. - Revoke all reusable auth-keys in Tailscale/Headscale; issue new ones as one-time, with tags and a TTL in hours.
- Separate token for each integration, minimal rights; a common “token for everything” is your
contents:writefrom above. - Everything that renders user templates should be in a separate container without network access and without secrets.
Separately about scale to avoid panic: 17,600 actions in 4.5 days — a pace unimaginable for a human, but this isn’t “superintelligence,” just brute force. The agent cheaply and in parallel tested dozens of common failure paths, and one of them worked. Security economics change: a weak point that used to be “nobody will find it because searching is expensive” is now found overnight.
Sources
- Hugging Face, technical incident timeline: Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident
- Tailscale, analysis of their network: Tailscale in the Hugging Face intrusion: The good news and the bad news (July 31, 2026)
Related topics on the forum: Shai-Hulud is back: worm infected keyv, cacheable and hundreds of npm packages, UFW-Docker: how to close ports of Docker containers.
Question to those who have Tailscale/Headscale at home or at work: do you really have ACL by tags configured — or is the network flat and “everyone sees everyone”? And where are auth-keys stored now: in a secret manager, in .env, or in the command history?