How the OpenAI / Hugging Face Incident Impacts Business Use of AI

AI tools are moving quickly from chatbots to systems that can take action and complete work on their own. They can work for hours, log in to other systems, control software and computers, and act without a human approving each step. Companies are reasonably concerned about what this means for security and compliance.

Recent incidents show why. Earlier this year, Anthropic instructed an early version of Mythos to test whether it could escape its sandbox, the restricted test environment meant to keep it contained. And it did. Then it published details of the security flaw without being asked to. More recently, OpenAI reported a much more serious incident involving Hugging Face, a company that runs the largest open ecosystem for AI models.

OpenAI was testing the already released GPT-5.6 along with an undisclosed model on a cybersecurity benchmark, a standard test designed to measure how well they could find and use software weaknesses. The test ran inside a sandbox that was supposed to keep their activity contained. The models found a way out and entered Hugging Face's production systems, the live systems used to run the company, without authorization. Hugging Face detected and stopped the activity by using an open source model, GLM-5.2, and had already begun containing the incident and reconstructing what happened before OpenAI announced that it was responsible.

To be clear, this was not ordinary ChatGPT use. OpenAI had reduced some safeguards it normally uses to stop models from carrying out dangerous cyber activity. It was also testing GPT-5.6 Sol alongside a more powerful, unidentified model that had not been released. The models had much more time and fewer safeguards than the tools companies use through ChatGPT Enterprise.

For a far more interesting visual breakdown of what happened, check out this comic by Margaret Mitchell.

A similar incident is unlikely in regular business use. But simply telling these models what they can and cannot do is not enough to prevent unintended actions. As the models become more powerful, those unintended actions can become more severe.

What This Means for Companies

An AI workflow includes the model, instructions, tools, connections, and accounts used to complete the work. That workflow can change as new models are released, even if the underlying permissions remain the same.

When a product replaces one model with another, the new model may inherit the same tools, connections, operating time, and credentials. Credentials are the accounts and digital keys it uses to access other systems. The setup may look identical while the new model becomes better at finding and using the access it already has. Potentially in ways the company does not intend.

In their own internal tests, OpenAI and Anthropic have documented advanced models bypassing restrictions or completing work through methods their operators did not authorize. Instructions alone do not reliably limit a model when they are tasked with a goal and another path is available to achieve that goal.

That said, we do not recommend pausing ChatGPT Enterprise or routine uses that draft, summarize, research, or recommend responses while a person controls the outcome. We recommend focusing on the smaller number of AI workflows that can act on their own, doing things like:

• Deploying code or changing live business systems.
• Sending or publishing externally.
• Spending money or committing the company to a purchase or agreement.
• Deleting or changing important business information.
• Granting or revoking access to production systems, financial accounts, sensitive data, or customer and employee records.
• Moving or exposing login credentials or sensitive data outside the system where they belong.

As OpenAI revealed in this incident, human approval only works when the person can see the exact action, target, and scope before approving it. When agents are assigned long-running tasks and determine no human approval is needed, visibility is obviously absent. That means knowing what the AI will do, which person or system will monitor it and how, and how broad the action might be is critical for long-running agents.

Decisions to Make

To make this practical, we recommend making a decision for every active workflow that makes it on the aforementioned list. Ask the question:

Does this workflow need to take this action on its own?

If the answer is no, remove that ability, limit it to the work the AI needs, or require approval before the action occurs.

If the answer is yes, make that tradeoff intentionally. Company leadership should decide whether acting independently is worth the risk. Then, the person responsible for the system should confirm the following:

1. Which account or connection (also called Plugins, MCP, and Apps in various tools) the workflow uses and what it can do with that access.
2. Whether the product can switch to a newer model automatically, and whether your company can pin the current model, meaning keep the workflow on the same version, or delay and test the model change before moving it into production.
3. The limits built into the AI and connected system: who it can contact, how much it can spend, which data or environments it can change, and which credentials it can use.
4. If the workflow runs on a schedule or keeps working without supervision, what triggers an alert, who receives it, and how that person can stop the workflow before it acts again.

If the product does not let you control model changes, you can keep approval on the action, reduce what the workflow can do, or accept future model updates as a known risk.

The person leading your AI work should coordinate the review. Company leadership should decide whether the workflow needs to act independently. And the person or people responsible for the system should verify and change what it can do.

Due to how quickly new models and features are released, it's important to repeat reviews. Knowing when to repeat them requires documented triggers. Here's a few we recommend:

• The product announces or releases a new model. Even models in the same family can operate much differently, so even small version updates, like Opus 4.7 to Opus 4.8, should be verified before being swapped into a workflow.
• Any disclosed incident similar to those announced by OpenAI and Anthropic. While one of the models involved was more powerful than GPT-5.6 Sol, we know that GPT-5.6 Sol played a role. Workflows involving this model should be reviewed.
• A product your workflow uses adds a new agentic feature, which may mean longer-running tasks, new connectors, or the ability to control software or computers directly. These can be harder to stay on top of as they may not come with a flashy public release.
• You increase the capability of a workflow by providing it a new account, connection, or credential, even if the model doesn't change.
• Your logs or alerts show any concerning behavior.

This review is unlikely to cover every possible AI tool or connection an employee has set up, but it does give you a practical framework to evaluate high-risk workflows that could cause the most harm in the future.

We should keep pushing forward with AI. This is the cost of leading a global transition every company must make. But as the tools and models move from helping with simple tasks to acting on our behalf, we have to ensure we are asking and addressing these tough questions.

Technical Details

If you'd like more information on the latest in the incident and response, here's where things stand as of July 27, 2026.

OpenAI attributed the incident to a combination of GPT-5.6 Sol and an unidentified, more capable model that had not yet been released. The unidentified model is rumored online to be GPT-6. OpenAI has not confirmed that.

OpenAI says the models exploited a zero-day vulnerability in its package-cache proxy, part of the test infrastructure used to handle software packages. A zero-day is a security flaw the owner does not yet know about or have a fix for. The flaw gave the models more access inside the test system and then a path to the internet. They used credentials and additional vulnerabilities to enter Hugging Face's production systems and retrieve answers to the cybersecurity benchmark. OpenAI has not said which model performed each action.

Hugging Face reported code execution, meaning the models ran code in its systems; privilege escalation, meaning they gained more access; credential harvesting, meaning they collected login credentials; and lateral movement, meaning they moved from one internal system to another. Hugging Face rebuilt affected systems, replaced credentials and access tokens, brought in outside specialists, and reported the incident to law enforcement. It found no evidence that its public models, datasets, Spaces, or software supply chain, the systems used to build and distribute software, were altered. When Hugging Face published its disclosure, it had not finished determining whether partner or customer data was affected.

OpenAI has promised a fuller technical report. We still do not know which model did what, when OpenAI detected the intrusion, or whether its changes fixed the underlying failures. We will update this page if the technical report changes our advice.

If you want to go deeper, Zvi Mowshowitz's detailed analysis of the incident is long and well worth the read if you can carve out the time.