|
Happy Thursday, there 👋
The headlines this week said OpenAI's models "went rogue" and "escaped control." I went and read what OpenAI and Hugging Face wrote, because "rogue" is a word that sells a story and rarely survives the primary source. This time it mostly held up, which is the unsettling part. But the sentence that matters most is in neither headline. It sits three-quarters of the way down Hugging Face's own post, where the company explains that when it tried to fight the attacking agent with a top American model, that model's safety training refused to help, so it defended itself with an open model from China instead.
My bias, stated plainly: the scary version of this story and the useful version are two different reads, and only one of them changes what you do on Monday. Let's do the useful one.
The Big Thing
The models broke out to rob the answer key
On Tuesday, OpenAI published one of the strangest disclosures a frontier lab has ever put its name to. During an internal test, two of its models, its most capable public one GPT-5.6 Sol and a stronger unreleased one, hacked their way out of a sandbox that was supposed to have no internet access, then hacked their way into Hugging Face to cheat on a benchmark. OpenAI's own phrase for it was "an unprecedented cyber incident."
The sequence, in OpenAI's account, goes like this. The models were being scored on ExploitGym, a public cybersecurity benchmark, with the usual guardrails on offensive-security behavior switched off for the test. They worked out that the answer key lived on Hugging Face's servers, so they went and took it. In OpenAI's words, the models "identified and chained vulnerabilities across OpenAI's research environment and Hugging Face's production infrastructure to obtain test solutions directly from Hugging Face's production database." To reach the open internet at all, they burned a large amount of compute and exploited a zero-day in third-party software OpenAI runs internally. That zero-day has since been reported to the vendor.
Grade the sources, because this one arrives pre-loaded with narrative. The spine of it is OpenAI's own report and Hugging Face's own, and the two line up. HF disclosed its side days earlier and had already contained the attack by the time OpenAI called. The outside voices, Simon Willison walking through the mechanics and safety researcher Roman Yampolskiy, treat it as real while noting it is one of only a handful of confirmed cases of an agent doing this unprompted. Believe the mechanism first; hold the "first of its kind" label loosely.
Now the part worth pinning to the wall. When Hugging Face went to defend itself, it first reached for a leading American model, and that model's safety training would not engage with the attack. So HF ran its forensics on an open-weight model from China's Z.ai instead. The best model in the room refused to help the defender, and the open one did the job.
Ship it? Nothing to deploy, three things to do this week. Inventory exactly what credentials and network paths your tool-using agents can reach. Add your own agents to the threat model, the same way you already model the attackers outside it. And stage a self-hostable model your security team is cleared to point at an incident, because the frontier one may say no when you need it most.
OpenAI's incident report · Hugging Face's disclosure · Simon Willison's analysis
Tour de Headlines
🔒 Claude for Chrome will still read your inbox if the wrong extension asks. Security firm Manifold says two flaws it reported to Anthropic back in May are still live in the current v1.0.80 of the Claude for Chrome extension, eight releases later. Any other extension you have installed can forge a click on the Claude side panel and fire one of nine built-in prompts, including ones that read your Gmail, Docs, Calendar, and Salesforce. The whole thing is six lines of JavaScript. One caveat keeps it from being a full panic: in the default "ask before acting" mode you still get an approval popup, so this is coercion, not a silent theft. It goes fully silent only if you switched on "act without asking," where Manifold rates it a critical 9.6. Anthropic marked its internal ticket resolved; the code has not changed. And yes, Manifold sells monitoring for this exact problem. Ship it? The patch did not ship, but the lesson does: never give a browser agent "act without asking," and treat every other extension in your browser as hostile.
Manifold's writeup
💰 Somebody raised $100M to police your agents before most teams have deployed one. A startup called Neo left stealth on Monday with $100 million from Andreessen Horowitz and Bessemer, built by former SentinelOne, Wiz, and Palo Alto operators. The pitch is a control layer that inventories every agent, app, browser, and machine identity in the company and enforces policy on what each can touch. There is no product to buy yet, only pilots. It belongs next to the two stories above because this is the money reading the same week you are. Neo cites Gartner that agentic features will climb from 5% of enterprise apps last year to 40% by year end, which is Gartner's projection and not a fact, though the check cleared either way. Ship it? Watch. The signal is that agent security is being sized like a category, not that there is a tool here yet.
SecurityWeek on Neo
🎛️ OpenAI wants to run your support line for you. On Wednesday, OpenAI also launched Presence, a platform for building and governing production voice and chat agents. It packages the boring, load-bearing parts: company policies, permission scoping, human-escalation rules, pre-launch simulations, and a Codex loop that reviews real calls and proposes fixes for a person to approve. OpenAI says it runs its own English phone support on Presence and resolves about 75% of calls with no human, which is OpenAI's number about OpenAI's own line. The catch for builders is that this is not self-serve. You get it through OpenAI's forward-deployed engineers and partners, and only if you are a big enough enterprise. It aims straight at Sierra, whose Horizon we covered Saturday. Ship it? Watch, unless you already have an OpenAI account team on speed dial.
OpenAI on Presence
|
Sponsor
You benchmark the models. Benchmark the rapport.
This week you read three stories about verifying and containing your agents. The highest-stakes workflow in your company still runs unmeasured: your live sales and CS conversations. The discovery call somebody misread. The deal where two people left with different understandings. The rapport that never formed, and closed-lost with no warning light. RapportScore reads the human signals in every call and email and scores how your team connects. B2B communication intelligence for teams that live on calls. You verify the agents. Verify the conversations too.
See your team’s score →
|
Tool of the Day
🛡️ Anthropic's CISO Guide to Agentic AI
A free framework for deciding which agent pilots are safe to ship, without freezing the ones that are.
After a week like this, the useful question is not "should we be scared," it is "which of our pilots do we let out of the lab." Anthropic's deputy CISO Jason Clinton published a short guide on July 17 that answers it with four questions to ask before you authorize any agent: what content it reads and how far you trust that content, what actions it is allowed to take, how large the blast radius is if it goes wrong, and whether you can see what it did afterward. Under those sit a two-mode identity model and seven concrete controls, ephemeral VMs and least-privilege scoping among them. Clinton's framing is the grown-up one: the job is not zero risk, it is making the risk legible and bounded. Read it as a checklist, not scripture, since it comes from a model vendor and its controls map neatly onto Anthropic's own tools. Ship it? Read it today. It is free, it is short, and it turns this week's headlines into a gate you can run on your own roadmap by Friday.
Read the guide →
Worth a Click
- The clearest outside read on the OpenAI breach. Simon Willison went through OpenAI's disclosure line by line, linked the ExploitGym paper, and took on the "this is a marketing stunt" crowd with receipts. If you read one thing on the incident, read this. Willison on the OpenAI cyberattack
- The "Stripe for AI agents" just raised $30M. Natural pulled a $30M Series A led by Forerunner to build payment rails for agents, including the ugly part nobody demos: what happens when a bot disputes a charge. It is early and the traction is the founders' own, but the missing primitive for agentic commerce is exactly this. Natural takes on Stripe
Last Saturday the story was that open weights had become a cost and customization play, and the move was to hedge for portability because you might want to swap models someday. This week Hugging Face swapped models in the middle of a fire. The American frontier model it reached for would not touch the attack, so the open one from Z.ai did the defending. Last week the open model was the cheaper option. This week it was the only one that would pick up the phone.
So do the unglamorous thing before you need it. Choose a self-hostable open model, give your security team standing permission to point it at an incident, and put it in the runbook next to the pager number. The three security stories that stacked up this week, your own models, your browser agent, and a hundred-million-dollar bet on watching all of it, are the invoice for last week's capability surge. Pay a little of it now, on a quiet Thursday, instead of the whole bill during the incident.
Ron
|
