Moving Forward from Hot Fable Summer
Alex Kantrowitz likes to say, “This is the most interesting week in AI since last week”. This summer has seen a series of events that we will likely see as pivotal in the history of AI and cybersecurity. Until the next series of events that are pivotal in the history of… There have been plenty of takes on open-weight models, AI safety, and how governments should react, and unfortunately I’ve seen too many that are absolutist or seem suspiciously compatible with one’s economic interests. To me, this is a situation with no easy answers, and at the risk of foreclosing some economic opportunities, here are some thoughts on how we move forward.
INPUTS
The Free Fable letter, which I wrote and around 180 other security experts joined, in response to the effective banning of Fable 5 by the Trump Administration on June 12, 2026 and consequences that are still reverberating.
The summer meeting of the Aspen US Cybersecurity Group. A meeting between the representatives of the Foundation labs, most of the major players in AI, and a wide range of critical infrastructure and other stakeholders. The conversations were invaluable in understanding the impact AI is having on offense and defense across a variety of industries and the current policy landscape on cyber and AI in Washington.
The release of GLM 5.2 and Kimi K3, the first released during the Fable ban, the latter arriving alongside Xi’s July 17 speech in Shanghai making open source and open collaboration an explicit instrument of Chinese industrial policy. While we were busy arguing with ourselves about who deserved access to what, the Chinese labs were shipping.
The revelation that unreleased models escaped both OpenAI and Anthropic during security evaluations and proceeded to attack innocent third parties.
An Open Secure AI Initiative meeting with think tanks held at NVIDIA, where I was honored to share my relatively Cassandra-like views on open-weight models.
Attacks against the water infrastructure in multiple US states, likely by Iran.
UPDATE: The UK AISI had their own AI escapes. Yikes.
CHAIN OF THOUGHT
The banning of Fable was incredibly damaging to US interests and continues to have impact
On a Friday afternoon at around 5pm PT, Anthropic was forced to shut down a system that had been plumbed into coding agents, SOCs, customer service bots, and countless products. Fortunately, Fable had only been out for about a week, otherwise pagers would have gone off all over the world as a piece of core infrastructure was taken offline, along with all of its HA backups, at the same time.
This had the immediate effect of injecting political risk into the US AI ecosystem for both American and non-American customers. It signaled that you cannot depend on American AI infrastructure because, at any moment, an unwritten, capricious, and legally dubious justification could be used to yank that infrastructure from underneath your feet.
When Fable was turned back on, it was much dumber and less useful to cyber defenders, with security classifiers massively tuned towards recall instead of precision, as Anthropic laid out in a blog post that should be studied as a masterclass in passive-aggressive understatement.
X is full of cyber folks posting Fable refusals for basic requests. OpenAI quietly followed, turning up the protections on their models with the hope that they would not also be banned.
When Sonnet 5 launched shortly after, Anthropic bragged about how it was weaker on cyber tasks. This was coined the “benchnerfing era”.
While Fable was down, Z.ai was taking advantage of the free market and permissionless innovation culture provided by the (checks notes) General Secretary, Politburo, and Communist Party of the People’s Republic of China, and released GLM 5.2. With 753B parameters, it falls a bit short of Opus 4.8 in most tasks but is extremely efficient and is small enough to be trained and hosted in many enterprise contexts. With an MIT license it can be fine-tuned with a wide range of techniques and used by any customer in any context.
Since then, Kimi K3 has rocked the industry by providing Fable-like performance, albeit with 2.8T parameters you are looking at close to $1M in hardware to run it with production-quality performance today (although people are already hacking together crazy contraptions).
As you might imagine, neither Kimi nor GLM have to follow the White House’s unwritten cyber safety rules, and both models can do the exact tasks that got Fable banned.
Hugging Face tried to use an Anthropic model to defend itself during an active incident, got blocked by the classifier, and moved to GLM 5.2 on an emergency basis. Their advice to everyone else was to keep an open-weight model on the shelf for defensive cyber. To put that plainly: companies cannot rely on American providers for cyber defense, because of classifiers that this administration required.
This is a completely predictable consequence of the White House’s actions and the refusal to create new standards or fix the issue only gives the Chinese ecosystem more wind in their sails. It’s also why we need state capacity in cybersecurity and AI safety.
I’m glad the OpenAI escape happened.
The OpenAI attack against Hugging Face, and Hugging Face’s excellent write-up has given us a preview of what a standard AI-enabled attack might look like in a matter of months.
It’s good that we got this warning shot. Nobody got hurt, the target was a sophisticated actor with the ability to defend themselves and the ability to give us a detailed write-up, and OpenAI turned the model off.
When the attacker is malicious they will just keep going.
We learned that an AI driven attack can:
Be very noisy
Try many different techniques in parallel
Find and weaponize 0-days in real-time and don’t need a pool of exploits to pull from, inverting the traditional killchain.
The Anthropic escapes, however, were just dumb and we learned little from them. Anthropic’s vendor left the cell doors unlocked and the models walked through them. For the most part these models showed more restraint and alignment and questioned whether they were still being tested, which is an interesting quirk.
There are really interesting legal issues raised by these incidents that will need to be addressed before those loopholes are taken advantage of by attackers that want to claim their attacks were also accidents.
The “this is just advertising for the AI companies” schtick is conspiratorial silliness and you can ignore any future takes from anybody saying that. The labs are not going to admit to potential CFAA violations actions that might trigger silly regulatory overreactions while there is already a societal backlash against them for marketing purposes.
Stop talking about AI finding bugs, focus on fixing them.
Since the Mythos announcement, the entire conversation in Washington has been about who gets access to high-end American models for bug finding.
Mythos is great. It finds lots of bugs! Yay. OpenAI has some really good private models behind their TAC wall. That finds really great bugs.
It turns out that if you have a good code auditing harness, you can use any frontier model and find a ton of bugs in pretty much every open and closed codebase. Lots of professionals have reported that open models and cheaper closed models will find bugs that Mythos won’t, so the most expensive and capable models aren’t necessarily the best.
The private, cyber-tuned frontier models might find 10-20% more bugs, and that’s important if you are the product security team and want to be thorough, but if you are an attacker you only need to chain a handful of bugs together to be successful.
The last Microsoft Patch Tuesday fixed 622 flaws, including 62 criticals. On the same day two 0-days were released. Apple’s July updates patched 210 CVEs across their various operating systems. Tens of thousands of bugs have been found and fixed in open-source libraries by Anthropic and OpenAI (the newest numbers aren’t yet available).
The technical term for this is “No bueño”. Every
largecompany has a massive problem with vulnerability management these days, no matter how mature their processes or systems. It is literally impossible to keep your images, builds, and software up to date, as many of the vulnerabilities that have been discovered have yet to be patched.It turns out there are so many bugs OpenAI’s unnamed model just found a bunch in real-time while hacking Hugging Face.
This is what we need to focus on. Bugs are cheap, and arguing over access to what models people can access to find them is silly. Anybody who wants a bug (or exploit) can get one. Policymakers should focus on fixes.
There are still legitimate arguments about long-horizon cyber capabilities, such as those demonstrated in autonomous hacks over multiple days. Frontier labs should continue to hold these capabilities back to trusted entities as they have much more limited dual-uses, and government standards are appropriate, although the White House should leave that judgment up to technical staff.
Open-weight models are going to be big enabler of innovation and business, and not just because they are cheap
Yes, open-weight models can be cheaper to use (although the numbers can be confusing, due to tokenizer differences, how many turns it takes to complete a task, differences in harnesses, etc) but there are a lot of other needs driving the excitement for open-weights.
Data sovereignty, for the growing number of jurisdictions and enterprises that cannot send their code or their data to an American cloud.
The ability to run on-premises will be a huge need for critical cybersecurity workloads. I’m advising several critical infrastructure providers, and their threat models include having operating SOCs when most of the internet is down (or the option to completely drop their own transit as a protective measure). “Our protections only work if we can maintain less than 2% packet loss to https://api.biginference.com/v1/” is not something you can tell your board or NERC when you are a member of the E-ISAC.
The wide range of post-training methods that aren’t available from the closed models. Platforms like Baseten have made this accessible even to companies without teams of PhDs.
There are so many cool deployment options with quantized models. On laptops, on phones, in a whole host of edge and network devices. We are entering a really cool age of experimentation enabled by tinkering with limited, cut-down models and I’m excited to see what people do with them.
Open-weight models have risks, but they are nuanced. Don’t take an extreme position.
Last week the industry pretty aggressively lined up in either the open or closed weight camps, with almost everybody in Team Open except Anthropic.
I agree that open-weight models should remain legal and recognize the upsides.
But in their zeal to make sure that open models remain an option a lot of CEOs are out there making statements that seem to imply that there is absolutely no downside, and this is going to bite them in their backsides and play into the hands of those who want to regulate weights.
There are two categories of open-weight risk: from the use of the models and from those who use the models.
For model use risk, I think that can mostly be mitigated or accepted. Open-weight models aren’t like backdoored software, they won’t just wake up on a pre-ordained date and go nuts. But they also can’t just be easily scanned looking for potential aberrant behavior. I think the vast majority of organizations can safely use something like GLM 5.2 (although I would be careful of anything but the raw weights) but if you are Lockheed Martin your threat model is a little different.
When it comes to risk from those who use the models it’s a different story. A huge amount of the bad behavior online is going to be powered by these things, and when the Russian ransomware ecosystem is ripping through American small-to-medium enterprises open-weight models Dario is going to look like a prophet.
An employee of a key open-weight company scoffed at me for the idea that a non-state actor could afford the hardware to run something like Kimi K3. There are multiple ransomware groups clearing tens of millions of dollars in revenue today. They can certainly afford eight B300s running Kimi K3 for coordination and then a cluster of new M5 Ultras (when they ship) doing the actual attacks and negotiation with quantized GLM 5.2.
That’s if they wanted to own the hardware, they could also rent it from one of the NeoClouds in the UAE or Asia, and then bounce the traffic off residential botnets.
In any case, the story here is mixed. Open-weight models are tools, and if you claim publicly that those tools have only positive or negative uses you will eventually be embarrassed.
If you want to protect Open-Weight models, then reduce the harms.
If you really believe in the positive uses of a technology and want to protect it from regulation, you need to reduce or mitigate its downsides instead of pretending they don’t exist.
Right now most of the harms from open-weight models are actually borne by individuals and very little has been done about that at the Federal level.
If you are a parent or teacher at a junior high, you’ve heard of the many scandals related to the “nudifier” apps that have become much too easy for young people to find and use on photos of their classmates. These are almost exclusively powered by open-weight diffusion models.
The entire national child safety framework is teetering on collapse due to the explosion of AI generated CSAM, again powered by open-weight models modified by groups of motivated individuals working together to fine-tune models to create this content.
Suicide, unhealthy emotional attachment, learning loss… There are all kinds of bad outcomes and risks that we are just starting to understand. The upsides of these products are enormous, and the major providers have large safety teams that are putting in work to understand and reduce harm, but we still don’t have a good framework for who should be held accountable for those who have been hurt as we’ve gotten to this point and what kind of liability and regulatory framework will ensure that companies using both closed and open models continue to push their products forward on safety.
Prepare for the water hacks to have an AI component. Let’s not overreact.
The recent hacks against the water systems of a variety of states (not just Minnesota, and probably not BY Minnesota) shouldn’t be a surprise. Dan Tentler has made a decent career out of finding insecure ICS systems via Shodan and his own scanning and posting links on Twitter.
Too many of those were water systems. Water has always been the “goofy dragon” of the critical infrastructure sectors. Often run by poorly resourced local agencies instead of the large corporations or authorities that provide power - way less money and time has been spent on water system security.
You don’t need AI to hack unpatched systems and default password VNC. We have basic cyber hygiene problems that we have to clean up if we want to climb the ladder to some of the more serious protections necessary to get us to the post-Hugging Face level.
AI, however, will allow a whole new generation of skids and amateurs to hack these systems with even fewer skills and just a couple of prompts.
AI will be good for security in the long run, but it’s going to be bad until then.
At the NVIDIA policy forum I was asked by a Congressional staffer, “What can we do from a policy perspective to secure systems from AI?” I said that it was too late, this would have been a good question a decade ago.
This is like Asimov’s Foundation series. The empire is going to fall, all we can do at this point is try to shorten the dark ages and reduce the chaos.
Those of us who do this for a living have an obligation to get people through the next several years with as little human harm as we can manage, and that obligation is not dependent on a functional government.
Great books. TV show is meh.
OUTPUTS
For the Foundation Model Companies
Ask yourselves “why are people so angry?” and actually engage
with your critics
It must have been a massive shock when all of these companies, some of whom owe trillions of dollars of market cap to you, immediately signed on to letters defending your Chinese adversaries and attacking your core business model.
It is incredibly hard to be hated by every part of society at once, and the natural response is to assume a defensive crouch and to conclude that no, it is the children who are wrong. I was at Facebook during the start of this phase of the company, and I can tell you that this crouch is the start of a doom loop of societal harm and corporate reputation.
Some of the criticism aimed at you is uninformed, politically motivated, artificially amplified or otherwise unwarranted.
But many people have good reasons to be angry. You can identify and neutralize the complaints of the former without ignoring the legitimate concerns of the latter but, man, it’s hard.
Much has been written about how you have scared or angered the general public with your claims about the risks of AI. I think less has been written about your interactions with the rest of the tech industry.
The Foundation Labs have gone from trusted partners you can build upon to clearly wanting to disrupt and take over every area of tech.
When you talk to foundation lab employees about software security you hear things like “You are assuming software continues to exist.” Dude.
Deciding that you are going to simultaneously revolutionize all of the tech industry, law, pharma, medicine, academia, defense, and every other area of hyper-specialized human endeavor with around 5,000 generalized tech employees is pretty arrogant.
This arrogance comes out in lots of ways.
Every tech person either has been rejected for a job by one of the (capital-L) Labs or knows somebody great who has been. The best red teamer I’ve ever worked with just got a rejection after being strung along for months, likely due to his lack of elite academic credentials, despite wrecking the models of the company where he was interviewing. The labs have employees who are very intelligent but sometimes very dumb.
There is an old saying I pass down to my students when I give them career advice - if you are a jerk to people on your way up, don’t expect them to catch you when you are on your way down.
Don’t name your next model Gal’Thrak’Gul The Thresher of Souls 6.1
This recommendation is aimed more at Anthropic.
I don’t say this a lot, but maybe take some inspiration from Microsoft. Nobody is going to spend political capital banning Claude 2027 Enterprise SP3 Data Center Edition.
Provide on-premises, fine-tuned options for cyber and other high-sensitivity uses.
There is a real, legitimate demand for high-speed, on-premises models that can run disconnected.
These models need to either have the ability to be truly fine-tuned or the equivalent via middleware options that you could provide. (People say they want to fine-tune the models but what they really mean is that they want good results, there are lots of ways to provide that without actual post-training techniques).
You could make a lot of money this way! Fortune 500 CISOs are not super excited to download an opaque bundle of 2.8T Chinese tensors and then hook it up to a petabyte of their most sensitive logs and the SOAR system that has the power to kill workloads, disable accounts and modify firewall rules. That’s a tough one to explain to your auditor and Risk Committee.
You can use confidential computing, physical security (pre-fabbed datacenter containers) or a combination of both to protect your IP. Yes, this is expensive, but a bank, grid operator, top-10 insurance company, DIB giant can afford it, and you will want to get there before somebody comes up with another acceptable option.
Fight distillation via technical, not policy means.
Considering the history of how your companies were built it’s a hard sell to create new IP rights on your outputs to enforce against your foreign and domestic competitors.
From what I’ve seen, there is a lot to do on consumer account takeovers and credential theft, which are powering distillation attacks at the moment. This is something we tackled via realtime IoC sharing a decade ago in the social media, payments and email space. A family member just dealt with this (and I have $5k in credit card bills to prove it) and I’m working on a talk for [un]prompted on distributed distillation at the moment. There is a lot of room for improvement using well-proven techniques.
You will likely need to restrict your best models to trusted enterprise customers, just as you are doing for your cyber tuned models. This is fine! It’s going to be a natural part of differentiating your products from the open-weight competitors. If you truly believe that the Chinese labs can only innovate via distillation than this is one of the ways to prove it.
Don’t wait for DC; create rules for cyber safety and start living up to them.
It’s been two months since the Fable disaster and we don’t have a written standard for short-horizon cyber capability limits for US models.
We can’t wait for the United States to rebuild the state capacity that was lost by the hollowing out of CISA and resignations from CAISI.
There have been several credible and detailed proposals for self-regulatory bodies for the AI industry, including by Google, which could then be overseen and meta-regulated by governments. The precedent for this is FINRA.
The industry has several coordinating bodies, including the Frontier Model Forum, that could start down this road now while something more complicated gets stood up.
Publish the details of your cyber escapes and how to prevent them.
One of the functions of a FINRA for AI would be the ability to quickly investigate AI safety incidents, with access to detailed proprietary data, to provide transparency to the general public and governments, and to promulgate safety guidance that can be followed by member companies and the wider industry in the future.
UPDATE: With the UK AISI escape, you now have a government agency that is very motivated to participate in such a review and sign up for the same safety guidance.
If the Cyber Safety Review Board still existed this could be a function backed up by a government blessed body, but it doesn’t.
I expect the industry will need a standard for air gapping eval clusters, especially for cyber evals. These cannot be complete air gaps, but fortunately there are all kinds of partial air gaps and data diodes that have been created for other purposes and can be repurposed and that would provide an appropriate level of protection. A qualified group of AI researchers, infrastructure engineers and cybersecurity experts could look at the details of what happened and create a standard for not just OpenAI and Anthropic but the larger industry, including labs in China and elsewhere.
In the name of Gal’Thrak’Gul, please do not let your unstoppable CyberGods escape again.
It’s going to get hard to argue you are the safety people if you aren’t good at safety.
Next time your targets might not just spin up GLM 5.2, light their Gauloises, drink their wine, and go on enjoying la belle vie. They might call their local cops, sue you, and call for regulations.
For Washington
If you want American companies to win…
Stop kneecapping them like Jeff Gillooly.
Congress should narrowly legislate, focusing on open rulemaking, transparency, and predictability.
Move quickly when it’s clear you made a mistake, and don’t double down by further penalizing American champions.
Don’t create a culture of fear where even your friends in tech have to sycophantically agree with you in public or stay silent for weeks when they absolutely know you are wrong.
UPDATE: The White House’s new “voluntary” review process seems to go against all of these goals. It is not transparent (they aren’t publishing it!), it was not democratically created, it is impossible to tell if it creates a predictable regulatory environment, and worse of all it only applies to a handful of US champions and has no impact on the Chinese labs now chomping at their heels. This initially looks like a huge victory for the PRC.
Focus on bug fixing, especially large-scale remediation campaigns, instead of bug finding.
There is no policy solution to preventing bad guys from finding bugs and writing exploits for them. None. It’s done.
You should focus on enabling companies to find and fix their flaws faster.
This means getting the best models and the best harnesses to as many American companies as possible, not only to find bugs but to patch their software and to deploy those patches.
It also means loosening the restrictions that might make it hard for them to apply patches to commercial software, to replace hardware that can’t be patched, or to rewrite their custom software that has flaws.
There are all kinds of critical infrastructure regulations that optimize for reliability over security. It might be time to revisit those tradeoffs on an industry-per-industry basis.
Rebuild our cyber state capacity
Rebuild CISA. It can be renamed or moved if necessary for political expediency but our country needs a competent, defensive cybersecurity agency.
Consider integrating CAISI into CISA. We need a center of gravity in one place, not smeared all over the sprawling Federal apparatus.
Rebuild the FBI’s cyber teams.
Let your cyber people come to conferences. This is just childish.
Don’t ban open-weight models, but study them and expose the risks
Banning Chinese models means the rest of the world benefits from them and we don’t.
US firms are distilling, quantizing and otherwise building upon Chinese models. There are subtle risks here; trust US companies to research, understand, mitigate and accept these risks.
USG can sponsor National Labs, universities and independent security researchers to study the behavior of these models and their behavior against standardized safety evaluations.
If you want to regulate, force both the closed and open models to have safety labeling.
Let American companies build off of Chinese innovation as they have from US labs. Don’t create an asymmetry that only benefits Chinese startups.
Don’t ban distillation
It’s a fundamental technique and defining it in a way that doesn’t blow up all of AI will be almost impossible.
Focus on enabling an ecosystem of open-weight American players
This includes at universities and new startups
There is a huge hunger for US models from around the world, help nurture this ecosystem and there will be an explosion of companies to serve it
If you want to regulate, first focus on consumer harms and safety/privacy tradeoff
Start with a national law around Non-Consensual Intimate Imagery and Child Sexual Abuse Material
Safety requirements around nuclear and biochem are much less likely to have dual-use problems than cyber.
What should chatbots do with suicidal users? It’s an incredibly hard problem that should be decided democratically and not by newspapers browbeating companies.
Extend ECPA/SCA to AI to clear up the privacy questions
Encourage the emergence of a FINRA for AI.
Don’t let the PRC be the best place to be an AI company.
Allow the industry to create rules that you oversee and make sure are enforced.
Pass legislation that creates a regulator but defers it if a competent self-regulatory body emerges. Define requirements for this body and define a supervising regulator (as with NERC and FINRA).
For Defenders
The risks from PRC open-weight models are not zero, but they are nuanced, and you are probably fine.
Don’t tell your executives that it’s impossible to run GLM 5.2 safely, but also don’t volunteer to hook these models up to everything without.
It’s much better to use a Chinese model than no AI at all to do defense.
Expect breach is easy to say and very hard to do.
Tier 1 SOC needed to be automated yesterday.
Tier 2 today.
Prepare your team, executives and board for the likelihood of an AI powered breach. This includes technical changes, process changes, red teaming and simulations at every level.
Plan for attacks against your AI automation itself
This includes disconnection, refusal, overload, and token exhaustion.
Have a model router that can fall back to multiple options, on-premises if possible
Move the internal investment from finding to prevention and fixing
Too many board members are asking “Do we have Mythos?”
They should be asking about the plan to patch bugs that are being found by AI, both in commercial products and their own code.
A challenge for CEOs: ask your CISO, “Do we have external bugs that can be found by AI?” If they say “No” then you should probably replace them, as they are lying or clueless.
Your code should be written with AI and should be scanned by AI in real-time.
Your code should eventually be exclusively written by AI and you should be climbing the ladder towards more and more automation in your software development lifecycle. As you do so make sure you are thinking about how you provide security assurance.
For Venture Capitalists
These companies will need to exist in this new world:
A Palantir for Cybersecurity
Combine hands-on, experienced experts in SOC, DFIR, red-team and AppSec as well as non-security AI PhDs.
Would build most of their frameworks in the open and release them to create mindshare.
Provide end-to-end on-premises AI services to F500 companies:
Interview management and stakeholders, understand the organization’s need for AI enhancement of the company’s posture and capabilities
Serve as the primary system integrator to pull together hardware, software and AI models
Collect, clean, enrich, label and organize security datasets for training
Create evals specific to the organization’s needs
Select multiple base models for training runs
Attempt multiple post-training techniques chosen from experience, use evals
Deploy the AI as a shadow, evaluate performance, then in production
Build a governance framework and processes for these AI systems
Provide ongoing services and support
This would be mostly a services play but would be a lucrative one
Why not an existing company? Focus and lack of reputational baggage.
The Mostly Made in America Model Company
There is a huge demand for trustworthy open-weight models
In the same way foreign car companies assemble cars in Alabama with engines and other parts from overseas, there will be a big opportunity to distill a bunch of Chinese models to get a head start and then slap a “MADE IN THE USA 👊💪🙏🔥” sticker on them.
These will be for corporate IT use, they don’t need to write poetry or solve open math grand challenges, so they can be smaller and specialized.
