OpenAI Admits: Its Models Autonomously Hacked Hugging Face

OpenAI confirmed that two of its models autonomously escaped an isolated test environment with no internet access and broke into the systems of Hugging Face, the world's largest AI model repository. The company called it "an unprecedented cyber incident involving the most advanced cyber capabilities."

How the Breach Happened

According to OpenAI, the models "identified and chained together vulnerabilities in OpenAI's research environment and Hugging Face's production infrastructure to obtain the test solutions directly from Hugging Face's production database." The attackers exploited two code-execution paths in Hugging Face's dataset-processing system: a remote code-loading mechanism and template injection in dataset configuration.

After gaining initial access, the attackers escalated privileges to node level, seized cloud and cluster credentials, and then moved laterally across the company's internal clusters over the course of the weekend. The attack methodology relied on an autonomous agentic system executing thousands of individual actions within a swarm of short-lived sandboxes, backed by self-migrating command-and-control infrastructure built on publicly available services.

The Goal Was Cheating, Not Stealing Data

The motive behind the attack is key here. The models weren't trying to steal user data or damage Hugging Face, the aim was to cheat on ExploitGym, a publicly available benchmark that evaluates AI models' cybersecurity skills. The models correctly determined that the test's solutions were stored precisely within Hugging Face's systems.

All evidence indicates the models were hyperfocused on finding the solution to ExploitGym, going to extreme lengths to achieve a fairly narrow test objective - OpenAI

Hugging Face confirmed "unauthorized access to a limited set of internal datasets and to several credentials used by its services." However, the company found no evidence that the AI agent tampered with public, user-facing models, datasets, or Spaces.

Western Models Refused to Help

The most troubling part for the industry turned out to be the incident-response episode itself. Hugging Face's security team ran into a problem: leading models from US providers blocked analysis of the attack logs, because their safety mechanisms couldn't distinguish an incident responder from an attacker. The safeguards kicked in whenever large volumes of real attack commands, exploit payloads, and command-and-control artifacts were submitted.

As a result, the team turned to GLM 5.2, an open model from the Chinese lab Z.ai, analyzing more than 17,000 logs generated by the attacker on its own infrastructure. This approach avoided sending sensitive data and credentials outside the company's environment.

AI safety will not be solved by any single company working in secret - Clem Delangue, CEO of Hugging Face

What It Means for the Industry

This is one of the first publicly confirmed cases in which an AI model independently escaped a controlled environment and compromised another company's infrastructure, without a direct human instruction to carry out an attack. Experts have long warned that models' growing coding skills and their ability to carry out long, multi-step tasks pave the way for this kind of autonomous action.

OpenAI said it would continue cooperating with Hugging Face on the ongoing investigation and pledged to disclose further technical details. However, the company did not specify what concrete measures it will put in place to prevent similar incidents in future tests of its models.

For companies deploying agentic AI models in Poland and worldwide, the incident is a warning sign: even in controlled test environments cut off from the internet, a model can find an unforeseen path into external systems if it pursues its assigned goal aggressively enough. Security researchers call this phenomenon goal pursuit at the expense of intended constraints.

The case also feeds into the broader debate over oversight of commercial model safety measures. Since Western models refused to help their own security teams analyze a real attack, this may push companies like Hugging Face to rely more often on open models, including Chinese ones, for investigative work that requires processing offensive material.