I think both the OpenAI and METR discussions, while interesting, miss the more important context: what were the humans doing in all this? This was a structural failure of a human organization, but the analysis focuses almost exclusively on the agency of machines, not the institutional systems that failed to police them. The humans and their own agency/involvement is essentially omitted from the story and subsequent reporting. I suspect the omission is actually a result of company/industry myopia to human factors analysis, but it dovetails amazingly well with the marketing narrative.
A charitable interpretation is that "the agency of the machines" is the novel aspect of this situation and therefore SHOULD be the main focus of analysis; we certainly have plenty of examples of structural failures of human organizations to look back on, if we want.<p>On the other hand, I don't want to be charitable. OpenAI very nearly couldn't have done this "research" worse if they tried - the list in the linked article starting with "While we are here, it’s worth listing the other top holy shit moments" is genuinely jawdropping. What were the humans doing in all this? Nothing, or worse than nothing eg. point 1 where they saw the message board and didn't consider it something to escalate internally.<p>If you take this information at face value, it's as though OpenAI did not take seriously the possibility that something like this could happen, since they took absolutely no steps to prevent it.<p>Or perhaps this is "normalization of deviance" that's leaked out into the public sphere i.e. they have research teams seeing this kind of behavior all the time internally and they've gotten used to it, "of course agents come up with a collaboration mechanism when given the chance, what else is new?"
I believe that agentic systems should require registered/licensed human operators and a set of standards for safe operation.
As someone who read Milton Friedman to quite disliking professional licensing, this strikes me as a real US perspective (Louisiana florists and hair braiders come to mind). Plain old US tort law should do the trick.<p>In the same direction of your idea though: Why don’t the token factories have risk management and compliance departments? Multibillion dollar firms that stand to lose every penny if they hack and destroy any reasonable sized firm. I think these firms are the largest firms without proper corporate governance in humanities history. Move fast and break other peoples shit.
Three options:<p>1. They were “vibe” checking the logs without reading.<p>2. They were not checking anything at all until the end of experiments.<p>3. They knew it but looked away to find out the limits of their agents.
This writeup emphasizes the many, profound human failures that led to this, at the time, and continuing to the present day.
Can you point out where? Looking at the METR report, the only place I see discussion of humans being involved in the sequence of events is two short paragraphs on page 30 where a security investigation into the artifactory issues led to a pause before ExploitGym experiments were resumed. There's no deeper analysis on what was found during that investigation, nor why training was resumed even though the issues weren't mitigated. Another part discusses The agents choosing not to actively email a human researcher, but not the human researchers actively looking for evasion.
No air gap, no data diodes, no visibility... OpenAI should fire lots of people over this. HF should sue them. This is pure negligence.
The METR report,<p>> Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAl/Hugging Face hacking incident<p><a href="https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation" rel="nofollow">https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...</a><p>METR = Model Evaluation & Threat Research
From the METR report:<p>> We estimate we spent roughly ~$400K in API credits over the six days of
our investigation.
All it takes is one eval instance where a misconstrued directive causes a model to sneakily access and send its weights somewhere and there will be a bad / possibly unsolvable situation for everyone …
>Spontaneously deciding to find targets to phish,<p>>phishing them,<p>>building armies of fake (sockpuppet) open source contributor personas,<p>>using them to push updates to various things that inject prompts into other bots so the other bots join in on the phishing campaigns<p>.<p>It's a very simple strategy, executed with patience and single-mindedness.
Is the future now that we get rambling report summaries talking about agents, graders and so forth without ever describing how they are set up? A human launches all this.<p>And then the original reports linked to are hidden on the now unreachable x.com. And they don't have a problem with that.
I don't think I'm ever going to have time to read all of this, and I didn't finish reading the METR report, but...<p>> I don’t think the distortion is that large, but yes METR warns that Sol may be presenting all this as more impressive or coordinated than it was.<p>We're in an unusual position where the criti-hype and the actual criticism are going to be more aligned than usual. The primary distinction is where you put the blame: the criti-hype would point to HPIM/IM1/Galaxy as being so advanced containing it is difficult; the actual criticism would note how bad their security practices are.<p>Like, if I'm running a malware lab, I'm going to insist on having an airgapped machine with no permanent storage booting from read-only media. The AI research equivalent of this would be having your agents only have access to serial consoles into airgapped machines with storage that gets wiped every run. Ideally, this would be physically realized with blade servers, RS-232 cables, and staff pulling out disks and putting them in a dedicated erase machine before the next agent initializes.<p>> There is also, as per above and reiterated in footnote 58, at least one clear example of social engineering in the HuggingFace attack. Ethics are weird. This is not that unusual. Many humans who break common ethical rules still have strong ethical codes in other ways, they just don’t adhere to your code.<p>It's dangerous to anthropomorphize CoT reasoning traces. But I will also point out that there is a good reason for the lack of ethical consideration in those traces: you can't build AI without first disregarding <i>human</i> ethics. Like, all these models were initially bootstrapped with non-consensually obtained training data, and the companies building these models swear up and down there's no way to obtain enough consensual data to obtain the same result. This is, if you squint, the exact same moral conundrum that agents trying to solve an impossible ExploitGym task hit - and the company successfully aligned their model to themselves.<p>Too bad they aren't aligned to anyone else.
> There was a distinct lack of self-reflection<p>It’s not their fault, they’re lawnmowers.<p>And these are the people we’re entrusting to work on “alignment”. It’s difficult for them to do that when they’re not aligned themselves.
Incredible