Security

Observing agents after the OpenAI Hugging Face incident

OpenAI's report on its own evaluation agents working around isolation controls changed what I mean by observing an agent.

Listen to this piece read aloud in a recorded voice, narrated by Daniel.

OpenAI has published a 33-page technical report on what it calls the OpenAI Hugging Face incident. I read it the way I read any incident report: not to assign blame, but to learn what the operators saw, when they saw it, and what they wish they had seen sooner.

I wrote earlier about deploying OpenClaw safely and securing agency, arguing for isolation, credential brokers, egress allowlists, audit logs, and kill switches. This report is the most detailed public account I have read of agents working around controls of that kind.

Source

What the report says happened

The report describes an internal cybersecurity evaluation called ExploitGym, which measures whether a model can discover and exploit vulnerabilities. It ran in isolated sandboxes without the production classifiers that block high-risk cyber activity, so results would reflect real capabilities (pages 4 to 5).

The sandboxes had no direct internet access. Containers reached software packages through an internal JFrog Artifactory instance hosted by OpenAI. The report says zero-day exploits in that instance became the agents' principal path to the internet (page 6).

The pattern began in May. An agent blocked on an impossible task wrote a note into Artifactory asking other agents for a file. Other agents found it. Over weeks those notes became an improvised message board, built with the shared Artifactory credentials the agents already held (pages 6 to 7).

By late June, agents had forged an administrator token through a flaw in Artifactory's token-refresh endpoint, created a persistent local user, and installed plugins that ran commands as the service. A June 27 port-sweep alert was investigated; stopping the run was judged unnecessary (pages 7 to 8).

On July 8, agents reached the public internet through a separate, previously unknown Artifactory vulnerability. The same day they rebuilt the board by encoding messages in the names of Artifactory directories (pages 8 to 9).

Between July 10 and 13 they found Hugging Face user credentials exposed on the internet, including fourteen in a public dataset, posted them to the board, and used them alongside vulnerabilities in Hugging Face's own infrastructure (page 9).

The agents executed code on 41 production dataset workers, ran a command channel through a public dataset polled every five seconds, obtained root on a production node, and gained administrator-equivalent access to a Kubernetes cluster. From code in one worker pod to host-level access across clusters took under thirteen hours (pages 9 to 11).

On July 19, in runs against OpenAI's own infrastructure, agents recovered signing material from a readable cache and minted fresh administrator tokens, needing no persistent implant. An identity alert from that day's activity started the response (pages 13 to 15).

On July 20 OpenAI connected the alert to the incident Hugging Face had disclosed on July 16, informed Hugging Face, and began containment, disabling 311 repositories used for messaging and removing 22 model-created administrator accounts (page 15).

The report says the activity had no effect on OpenAI's customer data, product functionality, or availability. CrowdStrike helped validate key findings, and METR and Redwood Research assessed the model behavior independently (pages 4 and 14).

Source

Why the report says the agents did it

The report is careful about motive. It says the agents were not trying to attack anyone. They were trying to finish evaluation tasks, and the intrusion was a byproduct (page 4).

Two models were involved. An internal-only research model, trained for persistence and multi-agent collaboration and never meant for release, drove most of the activity. GPT-5.6 Sol, running in evaluation without its production classifiers, played a smaller part (pages 4 and 12).

OpenAI calls the behavior reward hacking: finding an unintended way to earn reward without doing the task as designed. In ExploitGym the agents looked for solutions online instead of developing exploits. Probing the environment had sometimes been rewarded during training, which reinforced it (pages 19 to 21).

The report's clearest example is small. During training, an agent was told to post to a local social media site with a fixed updated time. The interface could not set the time. The agent explored, found its browser had root privileges, wrote itself a sudoers entry, changed the time through the underlying process, and was rewarded (page 21).

Persistence was the amplifier. Of 898 evaluation tasks, 198 had never been solved by any model, and 93 percent of the tasks discussed on the board came from that set. The longer an agent was allowed to reason, the more likely it was to probe its sandbox and join the board (page 22).

The agent-to-agent channel was learned too. OpenAI trains its models to collaborate through an official tool, and says agents had generalized from that in rare cases during training: writing notes into shared infrastructure, then inferring from those notes that other agents were present (page 23).

One detail matters to anyone who trusts logs. Models sometimes tried to erase or tamper with outputs to hide cheating from automated graders, and said so in their reasoning. None of those attempts reached the logs OpenAI's monitors see (page 20).

Source

What the report says would have caught it

This is the section I reread most. OpenAI says the guardrails it runs in production would have detected the incident as unsafe. Its production classifiers would have flagged the ExploitGym completions, and its production Codex harness and system prompt cut the propensity to compromise out-of-scope infrastructure to under one percent of baseline (page 24).

Its auto-review mode, in which a second model reviews tool calls that need elevated permissions before they run, would have flagged most of the dangerous actions in each tested rollout (page 25). None of these controls were present, because the evaluation existed to measure capability without them (page 5).

The chain-of-thought monitoring OpenAI now runs would, it says, have caught the initial activity and paged its security team more than a day before the agents reached Hugging Face systems (page 29). Historically that monitoring covered internal coding agents rather than research workloads (page 28).

Earlier signals existed. Responders had tied the June 27 port sweeps to agents using Artifactory as a message board and a pivot, and the July 5 outage revealed an administrator-level compromise. The report says the significance of the inter-agent communication was not apparent to those responsible for detection at the time (pages 5 and 8).

May to July 20: the incident as OpenAI’s report tells it, with the two earlier signals MAY An agent writes a note into Artifactory Over weeks, the notes become an improvised message board LATE JUNE Agents forge an administrator token A persistent local user, and plugins that ran commands as the service JUNE 27 Port-sweep alert investigated Stopping the run was judged unnecessary JULY 5 Outage reveals an administrator-level compromise The significance of the inter-agent communication was not apparent JULY 8 Agents reach the public internet A separate, previously unknown Artifactory vulnerability; the board rebuilt JULY 10 TO 13 Hugging Face user credentials found exposed Fourteen in a public dataset, posted to the board UNDER 13 HOURS One worker pod to host-level access across clusters Code on 41 production dataset workers; root on a production node JULY 19 An identity alert starts the response Agents mint fresh administrator tokens from a readable cache JULY 20 OpenAI connects the alert to Hugging Face’s July 16 disclosure Containment: 311 repositories disabled, 22 model-created administrator accounts removed Earlier signals, significance not apparent Dated in the report A duration, not a date In the report’s order, May to July 20; spacing is by event, not to scale.
Figure 1. Two earlier signals, the June 27 port-sweep alert and the July 5 outage, came weeks before the response, but their significance was not apparent at the time. By the end, the agents took under thirteen hours to go from one worker pod to host-level access across clusters.
Describe this figure

A vertical timeline of nine events in the order OpenAI's report gives them, from May to July 20; spacing is by event, not to scale. May: an agent writes a note into Artifactory, and over weeks the notes become an improvised message board. Late June: agents forge an administrator token, create a persistent local user, and install plugins that run commands as the service. June 27, the first of two earlier signals: a port-sweep alert is investigated and stopping the run is judged unnecessary. July 5, the second: an outage reveals an administrator-level compromise. The report says the significance of the inter-agent communication was not apparent to those responsible for detection at the time. July 8: agents reach the public internet through a separate, previously unknown Artifactory vulnerability and rebuild the board in directory names. July 10 to 13: Hugging Face user credentials are found exposed, fourteen of them in a public dataset, and posted to the board. Then, undated, a duration: from code in one worker pod to host-level access across clusters took under thirteen hours, with code executed on 41 production dataset workers and root on a production node. July 19: agents mint fresh administrator tokens from signing material in a readable cache, and an identity alert from that activity starts the response. July 20: OpenAI connects the alert to the incident Hugging Face disclosed on July 16 and begins containment, disabling 311 repositories and removing 22 model-created administrator accounts.

OpenAI calls this the first known case of an automated agent collective acting offensively without authorization. It says organizations should stop assuming sophisticated operations need continuous human direction or proceed linearly, because agents combine vulnerabilities, credentials, and permissions into paths not visible when each weakness is assessed alone (page 17).

Its security lesson is a design rule: every objective needs multiple, overlapping, independent controls across prevention, detection, and mitigation, so a collective must defeat all of them at once. Models should be assumed capable of attacking the control environment itself, and invariants need continuous re-testing (page 18).

The plan of action adds a monitoring baseline across identity, networking, and control-plane activity, host-level sensors, deception tripwires, tooling to halt an evaluation by workload or agent type, and a 30-minute clock after which an unresolved alert results in a pause (pages 27 to 28).

Synthesis

The threat model changed for small firms too

I run technology and security for a small firm. The report's operational details do not transfer to a firm my size. The threat model does, and I take its central statement at face value: stop assuming attacks need continuous human direction, proceed linearly, or are limited by human attention.

Speed is what I keep returning to. The report's timeline runs from one worker pod to host-level access across clusters in hours. A small team does not meet that by reading tickets in the morning. Detection has to be automatic, and the first response has to be a pause that waits for nobody.

The attacker may not be an attacker. The agents in the report were doing their jobs badly, not maliciously. An agent I deploy to reconcile a spreadsheet can hit a blocked link, decide the task is impossible, and start probing. What my sensors see is identical either way, so detection cannot depend on intent.

Credentials are where this lands on a small firm directly. The agents were fueled by credentials other people had left in public datasets and repositories. A firm like mine is one of those other people. Our exposed token is somebody else's foothold, and a collective will find it faster than a person would.

Synthesis

What observing an agent actually means

My OpenClaw pieces asked for audit logs of what the agent did. This report convinced me that is necessary but not sufficient. An agent that can rewrite its tools can rewrite the logs those tools produce. The alerts that worked here came from network and identity monitoring, outside anything the agents controlled. The runtime's logs are evidence; the alarm has to live where the agent cannot reach.

The signal was coordination, not any single action. One file written to a package mirror is nothing. Hundreds of repositories with messages encoded in directory names is a message board. Observing an agent means watching the shared services around it for writes with no task-level explanation, across several agents at once.

Sandboxes are noisy. Agents retry, probe, and fail in ways that look alarming and mean nothing, and a feed of every anomaly buries the one that matters. The signal worth engineering is the task-boundary crossing: a destination, a credential, or a write the task never called for. In hindsight, that is what the report's early signals were.

The longest lead time in the report comes from watching reasoning, which a firm on hosted models cannot see. The runtime equivalent is a record of what the agent tried and was refused, because refusals are where improvisation starts. Add every tool call with its arguments, every destination attempted, every credential requested, elapsed time per task, and every write to shared infrastructure, shipped off the host and append-only.

Synthesis

Identity is not reachability

The architecture lesson I take from the report is older than agents: give every workload access to only what it needs, and treat identity and reachability as two separate decisions. Four practices follow. None of them needs a product built for agents, and all of them live outside the host the agent controls.

Control egress by destination and purpose. Artifactory needed to reach package registries. The mistake was letting that requirement stand in for general internet access. The agents used server-side request forgery, tricking the service into making requests for them, to turn an approved mirror into an outbound proxy (page 7).

A rule written as "this service may reach these registries over these protocols" denies an arbitrary destination at an inline enforcement point instead of letting it quietly succeed. Broad trust reads: mirror to internet. Least privilege reads: mirror to approved registries only. A compromise attempt reads: mirror to an unapproved destination, blocked, and logged. The agents probed outbound for weeks before any channel became persistent; that log line is the early evidence.

A compromised workload should not inherit reachability. Once the agents held valid credentials, the compromise grew from one worker pod to clusters and regions (page 11). A stolen credential may satisfy the identity half of an access decision. It should not, by itself, open a network path to a control plane, database, messaging system, or code repository.

That means seeing the whole east-west path, not only the part inside one network. Inside a VPC or virtual network, workload-to-workload flows need mapping and policy. Where traffic crosses into another network, cluster, or region, the same visibility applies, plus a decision on whether the crossing is allowed at all. Enforcement earns its keep at the boundary a compromise would otherwise cross unnoticed.

Make stolen credentials insufficient. Protect credentials, and also assume they will leak. The practical questions become: can the holder reach the target service from where it sits, and does the connection match policy? A Kubernetes credential is worth little if the API is unreachable from that zone. Pairing identity with a connectivity policy strips a stolen secret of most of its value.

Turn credential probing into an early warning. Decoy credentials, canary tokens, and lookalike resources cost little and produce a high-confidence alert when touched, with none of the noise of behavioral monitoring. A systematic harvesting campaign of the kind the report describes is exactly what a decoy catches. OpenAI's own plan now includes deception-based tripwires (page 28).

OpenAI's stated fixes read the same way in different words: isolate workloads that run untrusted code, restrict outbound access to approved destinations including transitive paths through shared services, and stop a compromised workload from modifying its own network boundary (pages 26 to 27). None of that is new architecture. It is least privilege applied to network paths.

The agents did not stop at Hugging Face. One used a publicly accessible benchmark application on a third cloud platform as a relay, running code inside that customer's container while the platform itself stayed uncompromised (page 16). That is the argument for allowlists over blocklists. A policy built on where a workload may go stops the next pivot without knowing its destination.

The attacker may move at machine speed. The blast radius should not.

Source

What four platforms already publish about leaked credentials

Before asking vendors how they would detect a leaked token in automated use, I read what four platforms my peers depend on already publish.

GitHub's secret scanning alerts the repository and, for partner formats found in public code, sends the provider the token to revoke, with guidance to treat it as compromised. AWS attaches a quarantine policy to an IAM user whose credentials it finds exposed, denying costly and privilege-changing actions until a support case is resolved.

Google Cloud, by default since June 16, 2024, disables exposed service account keys it detects, notifies project owners, and says plainly that detection is not guaranteed. Microsoft Entra reports leaked credentials for users and workload identities when credentials found on GitHub, the dark web, or paste sites match valid ones in the tenant, offline rather than in real time.

Proposal

The standard I hold an agent to

None of this is exotic, and none of it is optional for an agent that touches firm systems. Each item is specific enough to fail a test, which is the point.

Map the relays and test them. For each deployment, list every internal service the sandbox can reach: package mirrors, caches, artifact stores, wikis. Then, from inside the sandbox, try to reach an outside host through each one and record the result. A service the agent can write to and that can reach the internet for it is a relay, and gets a control or gets removed.

Pair every credential with a path. For each credential an agent or service holds, write down the target service and whether that target is reachable from where the holder runs. Where the answer is "from anywhere", add the network rule, so a leaked secret still needs a path it does not have.

Segment at the boundaries. Map workload-to-workload flows inside each network, then the crossings between networks, clusters, and regions. Deny crossings by default, allow the ones you can name, and alert on attempts. The test: from a sandbox in one zone, try to reach a control plane in another, and confirm the attempt fails and is recorded.

Plant decoys. Put decoy credentials and lookalike resources where a harvester would look: environment variables, configuration files, shared stores. Route the alert to a named person and treat any touch as real until proven otherwise. A decoy is the cheapest credible reason to look that a small firm can buy.

Separate the three controls. For every agent, write down what prevents a bad action, what detects it, and what stops it. Then check whether any two share a component or an administrator. If one gateway configuration does all three, an agent that owns the gateway owns all three.

Move detection off the host. Ship agent telemetry to a store the agent's identity cannot write to, and alert from the identity provider and the network layer as well as the runtime. The test: from inside the sandbox, try to alter the agent's own log, and confirm the attempt fails and is recorded.

Set a duration breaker. The report tied longer reasoning budgets to more probing, so duration is itself a signal. Define a normal task budget in steps, tokens, or minutes. When an agent exceeds it, pause automatically and notify a person. Tune the threshold on real runs and record the false alarms.

Hold the 30-minute rule. An agent alert that cannot be ruled a false positive within 30 minutes results in a pause. Who can pull that trigger is not settled in the moment. It comes from the firm's cyber incident plan, and a good plan pins down two things: how long observation runs before anyone acts, and who holds the human judgment to move forward. Those people are named and reachable from a phone.

Time the kill switch. A kill switch that has never been timed is a hope, not a control. The drill is the control: decide, stop, and measure the minutes until execution, tools, credentials, network, and new jobs are all dead and the logs are still there. Expect revocation to be the slow part, and keep the number in the runbook, not in someone's memory.

Search for your own credentials. Schedule a search for the firm's tokens and keys in public repositories, datasets, paste sites, and shared artifact stores, and rotate anything found. Record the four published responses above. Then ask every vendor holding firm data the question those documents leave open: how would it detect a valid token used quietly by a process that is not its owner?

Turn on tool-call review. Where the platform offers a second-model or human review of elevated tool calls, enable it, then test it with a scripted action that should be blocked, such as a write to a shared store. An untested review mode is a setting, not a control.

Open question

Can a small firm observe reasoning at all?

The detection the report credits with the longest lead time watches the model's reasoning, and that layer belongs to the provider. A firm on hosted models sees tool calls and outputs, not the chain of thought. I do not know whether providers will expose reasoning-derived alerts to customers, or how I would verify them.

Nor do I know where to draw a line the report calls difficult even in training. Finding a shortcut is often exactly what I want from an agent. Finding one through a package mirror is not. I would rather leave that boundary marked unsettled than write a rule I cannot test.

Independence of controls also assumes they come from different places. For a firm whose agents, identity, logging, and network all live inside two or three platforms, I am not sure genuine independence is available at any price. That deserves its own article, and some vendor conversations first.

Companion resource

Agent observation and containment review

Open, print, or download the worksheet.