Just as a moderating comment - to detect many of these historically present vulnerabilities, Claude requires access to the source code - which in most cases is not public or available to hackers. It can, of course, also find vulnerabilities in running code, but not to the extent it can from the source. One would assume that all significant open-source code would be analyzed quite soon, and than subsequent releases would also be analyzed.
This is not to say that Claude doesn't represent a significant threat. But it can also be looked at as a resource that will, over time, reduce that threat.
90%+ of every piece of modern software is open source software, much of which you can glean from things like their OSS disclosures, documentation, and known functionality. The source code is already out there.
I'm an attorney specializing in OSS and can tell you that something like a data analytics platform can easily have 12k+ individual OSS components. More if they containerize. And thousands of customers can integrate that platform into their own platforms and so on and so forth. Imagine a company like VISA and how many giant platforms make up part of their workflow and/or have access to their data or their systems. It's likely that more than a million pieces of OSS touch their data or networks in one way or another - everything from fraud detection software, CRM systems, router firmware, to the latest TikTok knockoff some temp just downloaded onto their phone. You do not need to get to the heart of Visa's payment software to upend 60% of transactions in the US.
Problem is that a huge amount of open source is either not actively maintained at all or maintained by a skeleton crew already sweating bullets trying to make updates. The corporate world doesn't have the capacity to fork all those projects and properly harden them and they currently see no real urgency in trying to fund the existing maintainers.
Fundamentally, we have a Jenga block stack of software and if a company wants to truly secure it, that means working with a lot of third parties, each of whom own a little block, and huge coordination problems. For example, maybe company X could fund a handful of projects- but maybe they decide they'll wait to see if Google does it first. See how that works?
I will also say - I have some visibility to the administration of a large open source code base with a well funded vulnerability program. What Mythos does is not entirely new - the curve bent about a year ago where the bulk of reported vulnerabilities to our project are now AI-generated and submitted, to the point that it is overwhelming human abilities to even triage.
I'm optimistic in the steady state that we will be in a better place as legacy exploits get addressed and new ones are caught by these tools before they are added to the codebase, but it's going to take some time to get there.
The private governance frameworks point is where this gets really hard in practice. The audit model makes structural sense but the institution that does it has to solve difficult design problems simultaneously: enough independence to be credible to the public, enough technical access to be credible among the labs, and meaningful enough to make participation rational for companies that have no obligation to cooperate.
Even then, no frontier lab will volunteer for that level third-party scrutiny unilaterally; the competitive exposure is too asymmetric. You need all the major players to move together, and that only happens when there's a credible forcing function from above. Someone has to set the table before anyone will sit down.
The hard part here is not holding back Mythos itself, but in actually doing something about the long-tail of vulnerable deployments. It's all well and good for the highest risk companies (and government agencies) to get a preview and a head start. But I think it's fundamentally misunderstood outside of IT professionals (and many times within), how long the deployment tail is: https://substack.norabble.com/p/deployments-cant-wait. That's in many ways been one of the long running crises in IT for decades. And while it's received some attention, it's also received a lot of avoidance and misdirection.
As Arod suggests it's time to think beyond individual countries. You said it yourself when describing one of the precepts of SB 53:"... the framework rests on the notion that AI will not be controllable--that stopping the diffusion of potentially dangerous capabilities is impossible--and that therefore today’s “frontier” capabilities will be broadly dispersed within a short while." Rather than suggest that China and the US (and everybody else) now have a mutual interest in coming up with an international architecture to manage ungoverned diffusion of AI you double down on "preparing defenses." Can we get ahead of a catastrophe for once?
There will certainly be more lag on the compliance side. Compliance frameworks haven't processed either half of that equation. Auditors are still accepting annual pen test results as evidence of control maturity, a control designed for a world where sophisticated vulnerability discovery was expensive and scarce. The organizations signing off on that posture aren't deceiving anyone. They're operating on assumptions that became structurally false last month.
The framing that "looking hard" used to be expensive is the clearest way I've seen anyone articulate the actual shift. The security community has talked about offensive/defensive asymmetry for decades, but the cost collapse you're describing is different in kind. A zero-day used to require a world-class researcher and weeks of focused work. If Mythos-class models compress that to hours at API pricing, the economics of vulnerability stockpiling change completely. Nation-states that previously rationed zero-days for high-value targets can now afford to burn them on mid-tier infrastructure. The "transitional period" Anthropic describes optimistically could look a lot more like a permanent condition for any organization that can't afford continuous AI-powered defense. And most of the world's critical infrastructure operators can't.
These models are starting to feel less like tools and more like systems with dynamics we only partially control, something closer to large-scale natural systems.
Not quite hurricanes. We still decide when and how they form.
But once deployed at scale, their effects propagate faster than our ability to contain them. From there, we’re in the business of forecasting, hardening, and absorbing shocks, whether we’re ready or not.
“I am done with tiptoeing now.” Reality bites. At some point, the chickens would come home to roost and the AI resisters (who have made something of a cottage industry around critiquing AI from all angles) would be faced with an undeniable wind. Merely clutching their strongly worded anti-AI screeds would be insufficient to meet the challenge. Now we’re here and you no longer have to worry about speaking softly.
The fact that it is desirable to have some kind of reporting mechanism of potential risks posed by advanced AI models doesn't support the mechanism in SB 53, just the opposite.
The SB 53 mechanisms don't produce the kind of incentives to get the most accurate estimates of risk while producing the best possible capabilities. Exactly because of their public nature they encourage companies to minimize the dangers or not to investigate how robust their safeguards might be.
I absolutely think the us government needs to be kept aprised of risks for the reasons you say but SB 53 is to AI what environmental impact statements are to government construction.
"No guardrails on the open plains!" Indeed. I always take strength from your commentary, keep it up. I hope America can, as Churchill supposedly said 'be trusted to do the right thing, once they've tried everything else'
Dean, thanks for coming to HKS this week. This is another excellent piece. While I am a bit more skeptical and cynical about Anthropic motivation, I fundamentally agree that they are approaching it the right way, and I would rather have them in the lead than OpenAI, which has proven to be much more opportunistic.
I do not see that "winging" this one is a sound strategy, and we should be beyond the "Collingridge dilemma" debates. It is time for action. There are multiple ways to tackle governance that will not put the U.S. at a competitive disadvantage - from IVOs, to dynamic governance models, to some of the state-level experimentation. Let's do it.
From the peanut gallery, what, if anything, can individuals do for self/-defense? Hoi polloi like myself employ strong passwords, 2FA, VPN, etc., but these, I guess, will be like digging foxholes against ballistic missiles. Are we simply at the mercy of labs and software giants?
Using unique passwords is like making sure to be downstairs when a nuke falls on you, but it *is* helpful. The big deal is going to be pushing companies to respond and keeping your general life as resilient to disruption as possible. Not much else to do.
I was very much with you until that last piece. Maybe its me not being an american, but i think it is much of a Western / global issue than it is american. I hope America wings it, but at the same time I hope more societies will be a part of winging it (i fully agree that we are at the point of winging it).
You can also count on it that there are at least 20+ Israeli cybersecurity teams working on it right now. With a challenge like Mythos you'll need them too.
Dean, love reading your thoughts on this. As a mathematical biologist who contended with the dangerous gain of function EO and also developed AI tools for national security applications, I feel there are major opportunities to do good in tech policy these days.
Would love to chat more sometime - I believe there are common threads across tech policy that could help us accelerate across tech areas without undermining NatSec. Lmk if you’re keen, otherwise look forward to reading your thoughts.
Just as a moderating comment - to detect many of these historically present vulnerabilities, Claude requires access to the source code - which in most cases is not public or available to hackers. It can, of course, also find vulnerabilities in running code, but not to the extent it can from the source. One would assume that all significant open-source code would be analyzed quite soon, and than subsequent releases would also be analyzed.
This is not to say that Claude doesn't represent a significant threat. But it can also be looked at as a resource that will, over time, reduce that threat.
90%+ of every piece of modern software is open source software, much of which you can glean from things like their OSS disclosures, documentation, and known functionality. The source code is already out there.
I'm an attorney specializing in OSS and can tell you that something like a data analytics platform can easily have 12k+ individual OSS components. More if they containerize. And thousands of customers can integrate that platform into their own platforms and so on and so forth. Imagine a company like VISA and how many giant platforms make up part of their workflow and/or have access to their data or their systems. It's likely that more than a million pieces of OSS touch their data or networks in one way or another - everything from fraud detection software, CRM systems, router firmware, to the latest TikTok knockoff some temp just downloaded onto their phone. You do not need to get to the heart of Visa's payment software to upend 60% of transactions in the US.
Problem is that a huge amount of open source is either not actively maintained at all or maintained by a skeleton crew already sweating bullets trying to make updates. The corporate world doesn't have the capacity to fork all those projects and properly harden them and they currently see no real urgency in trying to fund the existing maintainers.
Fundamentally, we have a Jenga block stack of software and if a company wants to truly secure it, that means working with a lot of third parties, each of whom own a little block, and huge coordination problems. For example, maybe company X could fund a handful of projects- but maybe they decide they'll wait to see if Google does it first. See how that works?
I will also say - I have some visibility to the administration of a large open source code base with a well funded vulnerability program. What Mythos does is not entirely new - the curve bent about a year ago where the bulk of reported vulnerabilities to our project are now AI-generated and submitted, to the point that it is overwhelming human abilities to even triage.
I'm optimistic in the steady state that we will be in a better place as legacy exploits get addressed and new ones are caught by these tools before they are added to the codebase, but it's going to take some time to get there.
The private governance frameworks point is where this gets really hard in practice. The audit model makes structural sense but the institution that does it has to solve difficult design problems simultaneously: enough independence to be credible to the public, enough technical access to be credible among the labs, and meaningful enough to make participation rational for companies that have no obligation to cooperate.
Even then, no frontier lab will volunteer for that level third-party scrutiny unilaterally; the competitive exposure is too asymmetric. You need all the major players to move together, and that only happens when there's a credible forcing function from above. Someone has to set the table before anyone will sit down.
The hard part here is not holding back Mythos itself, but in actually doing something about the long-tail of vulnerable deployments. It's all well and good for the highest risk companies (and government agencies) to get a preview and a head start. But I think it's fundamentally misunderstood outside of IT professionals (and many times within), how long the deployment tail is: https://substack.norabble.com/p/deployments-cant-wait. That's in many ways been one of the long running crises in IT for decades. And while it's received some attention, it's also received a lot of avoidance and misdirection.
As Arod suggests it's time to think beyond individual countries. You said it yourself when describing one of the precepts of SB 53:"... the framework rests on the notion that AI will not be controllable--that stopping the diffusion of potentially dangerous capabilities is impossible--and that therefore today’s “frontier” capabilities will be broadly dispersed within a short while." Rather than suggest that China and the US (and everybody else) now have a mutual interest in coming up with an international architecture to manage ungoverned diffusion of AI you double down on "preparing defenses." Can we get ahead of a catastrophe for once?
There will certainly be more lag on the compliance side. Compliance frameworks haven't processed either half of that equation. Auditors are still accepting annual pen test results as evidence of control maturity, a control designed for a world where sophisticated vulnerability discovery was expensive and scarce. The organizations signing off on that posture aren't deceiving anyone. They're operating on assumptions that became structurally false last month.
The framing that "looking hard" used to be expensive is the clearest way I've seen anyone articulate the actual shift. The security community has talked about offensive/defensive asymmetry for decades, but the cost collapse you're describing is different in kind. A zero-day used to require a world-class researcher and weeks of focused work. If Mythos-class models compress that to hours at API pricing, the economics of vulnerability stockpiling change completely. Nation-states that previously rationed zero-days for high-value targets can now afford to burn them on mid-tier infrastructure. The "transitional period" Anthropic describes optimistically could look a lot more like a permanent condition for any organization that can't afford continuous AI-powered defense. And most of the world's critical infrastructure operators can't.
These models are starting to feel less like tools and more like systems with dynamics we only partially control, something closer to large-scale natural systems.
Not quite hurricanes. We still decide when and how they form.
But once deployed at scale, their effects propagate faster than our ability to contain them. From there, we’re in the business of forecasting, hardening, and absorbing shocks, whether we’re ready or not.
“I am done with tiptoeing now.” Reality bites. At some point, the chickens would come home to roost and the AI resisters (who have made something of a cottage industry around critiquing AI from all angles) would be faced with an undeniable wind. Merely clutching their strongly worded anti-AI screeds would be insufficient to meet the challenge. Now we’re here and you no longer have to worry about speaking softly.
So Mythos discovered some zero-days.
Zero days are discovered with reasonable frequency, so this is not nearly as devastating as the hype would have you believe.
* A man walks up to you and, more quickly than you can respond, draws a knife and stabs you *
"So you're bleeding. You bleed small amounts with reasonable frequency. This is not nearly as devastating as the hype would have you believe"
Didn’t it discover thousands of them?
The fact that it is desirable to have some kind of reporting mechanism of potential risks posed by advanced AI models doesn't support the mechanism in SB 53, just the opposite.
The SB 53 mechanisms don't produce the kind of incentives to get the most accurate estimates of risk while producing the best possible capabilities. Exactly because of their public nature they encourage companies to minimize the dangers or not to investigate how robust their safeguards might be.
I absolutely think the us government needs to be kept aprised of risks for the reasons you say but SB 53 is to AI what environmental impact statements are to government construction.
"No guardrails on the open plains!" Indeed. I always take strength from your commentary, keep it up. I hope America can, as Churchill supposedly said 'be trusted to do the right thing, once they've tried everything else'
Dean, thanks for coming to HKS this week. This is another excellent piece. While I am a bit more skeptical and cynical about Anthropic motivation, I fundamentally agree that they are approaching it the right way, and I would rather have them in the lead than OpenAI, which has proven to be much more opportunistic.
I do not see that "winging" this one is a sound strategy, and we should be beyond the "Collingridge dilemma" debates. It is time for action. There are multiple ways to tackle governance that will not put the U.S. at a competitive disadvantage - from IVOs, to dynamic governance models, to some of the state-level experimentation. Let's do it.
From the peanut gallery, what, if anything, can individuals do for self/-defense? Hoi polloi like myself employ strong passwords, 2FA, VPN, etc., but these, I guess, will be like digging foxholes against ballistic missiles. Are we simply at the mercy of labs and software giants?
(Yes)
Using unique passwords is like making sure to be downstairs when a nuke falls on you, but it *is* helpful. The big deal is going to be pushing companies to respond and keeping your general life as resilient to disruption as possible. Not much else to do.
I was very much with you until that last piece. Maybe its me not being an american, but i think it is much of a Western / global issue than it is american. I hope America wings it, but at the same time I hope more societies will be a part of winging it (i fully agree that we are at the point of winging it).
You can also count on it that there are at least 20+ Israeli cybersecurity teams working on it right now. With a challenge like Mythos you'll need them too.
Dean, love reading your thoughts on this. As a mathematical biologist who contended with the dangerous gain of function EO and also developed AI tools for national security applications, I feel there are major opportunities to do good in tech policy these days.
Would love to chat more sometime - I believe there are common threads across tech policy that could help us accelerate across tech areas without undermining NatSec. Lmk if you’re keen, otherwise look forward to reading your thoughts.
Does this mean hackers and software busters will go out of business?