Aviate, Navigate, Communicate

What to do about Mythos

Dear readers,

I have exciting news: I am writing a book! It is currently untitled and will be published next year by Penguin Press. The book will be a work of political theory just as much as it will be a book about AI. I cannot wait to tell you more about it. All I can say is that it is the most ambitious project I’ve ever undertaken, and an effort to answer—tentatively—questions I first asked here. Practically speaking, this does mean I expect my publication volume to drop, though I do not know how much. I expect a minimum of two essays per month, and hopefully more.

It is because of you—spreading my work by word-of-mouth—that this project exists. Thank you.

Dean

—

Ideas diffuse through society in a wavelike fashion, much like a rock thrown into a lake. Those closest to where the rock meets the water experience the wave first; they are the first-comers. Then the wave works its way out, reaching a wider radius with time. Your exposure to an idea depends just as much as where you are in the pattern of diffusion as it does on how “smart” you are (though: where you have situated yourself may indeed correlate with your intelligence).

Awareness of AI as a genuinely transformative technology—not as the next “internet platform technology” but as something deeply more powerful than that—comes in waves, just like any other idea. The big difference between this metaphorical wave and a real wave is that real waves weaken with time; the AI wave, by contrast, gains amplitude as it diffuses. When the wave hits you, it can be shocking. It is the moment when you realize that of course there is going to be a major role for the state to play in the future of AI, and that some portion of that role will come from the most restrictive and inhibitory ward of the state—the national-security apparatus. Getting hit by this wave is not quite the same thing as being “AGI-pilled,” but it surely sets one on that path.

Like all waves, there will be multiple peaks and troughs. But a few weeks on, and surveying the public and private landscape in Washington, it is clear that Mythos—the as-yet non-public model from Anthropic with extraordinary hacking capabilities—has become one such wavelike moment. It will not be the last. But right now, a whole host of newcomers—some of them quite powerful—have arrived on the scene. They have been hit by the wave, and so they have taken notice of “advanced AI” and its risks. As a result, a reset in AI policy and politics appears to be underway. Its direction, however, remains up in the air.

Thus far, senior Trump Administration officials have systematically underrated the potential for catastrophic risks from frontier AI in their rhetoric. I have argued that ignoring those risks is both a substantive mistake and a political one. It is a substantive error because catastrophic tail risks are often not mitigated well by markets or market-ish incentives like tort liability; risks of this type are classic examples of areas where deliberate regulation is often necessary. And it is a political mistake because it is simply untenable to pretend as though all AI regulation is equally bad, and that therefore any AI law is damaging. It was clear that, once AI’s catastrophic risk potential manifested itself in a sufficiently photogenic way, policymakers, business leaders, and the public would demand action. Indeed, a major fear of mine has been that, once that moment happened, policymakers would panic and push for a much more stringent regulatory response than I believe is needed.

The good news is that denying the problem may well have finally died as a strategy, at least among policymakers. In an interview last week, former White House AI Czar David Sacks was clear that he thought models with novel and potentially dangerous capabilities, like Mythos, could not be made available to the public as soon as they are ready. They require some sort of staggered release, he seems to argue, a process in which the government presumably has a role.

The bad news is that panic and overreaction are live risks. The White House AI policy staffers who fear regulation are not wrong; their fear is just exaggerated. The hard part is to avoid both overreaction and underreaction, to handle the slippery eel that is frontier AI without letting it slide out of your hands. We have seen the failure mode of underrating advanced AI from how out of step the White House found itself earlier this year when they learned of Mythos. But overreacting is just as much of a mistake; in fact, overreaction has the potential to be much more damaging.

—

Why should you not overreact to Mythos?

Mythos finds security vulnerabilities in software systems, often by stringing together multiple distinct bugs into one coherent exploit. These vulnerabilities were not created by Mythos but rather found by it. Previously, only elite humans—sometimes employed by large technology firms or government intelligence agencies, sometimes operating independently in a software exploit black market—could find vulnerabilities in this way. Mythos means that the cost structure of finding these vulnerabilities has shifted dramatically; instead of having to employ an elite human, large (and growing) fractions of this job have been automated.

Corporations who maintain software have a clean incentive to find and fix vulnerabilities. They are almost always “cyber defenders.” But governments, contractors who sell hacking services to governments, and criminal organizations have more complex incentives. Criminals are almost purely cyberattackers; contractors are mercenaries, selling services to both attackers and defenders. And governments themselves are hybrids, engaging in both cyberoffense and defense operations.

Like large software corporations, governments—especially highly capable cyber actors like the U.S.—expend significant resources finding or purchasing vulnerabilities in software. But unlike corporations, governments sometimes keep those vulnerabilities to themselves because they want to use them for espionage or cyberoffensive purposes. Corporations will use models like Mythos almost entirely for cyberdefense; it is impossible to say the same for governments.

Governments are therefore the sole wholly legitimate actors in society who have an incentive to find, hide, and exploit cyber vulnerabilities. They do so for at least ostensibly legitimate national-security purposes, but the fact remains: governments have an incentive for the world’s software to be less secure in a way that no other legitimate actor in the world does. This is not intended as a critique of governments—so long as cyberoffense exists at all, I want a government with a powerful cyberoffensive capability. Instead, it is intended as a warning against overreaction to Mythos. Any overreaction is likely to manifest itself in policy and regulation—in other words, in state control. An overreaction to the security risks of Mythos and similar models is therefore liable to hand more control to the sole legitimate actor who has an incentive to use Mythos to make the world less secure rather than more secure. This is unwise especially if you are concerned about the security risks Mythos creates.

Of course, and even more importantly, Mythos also improves U.S. cyberdefense vis a vis foreign adversaries. For many years, the United States government has maintained an interagency coordination group called the Vulnerabilities Equities Process which decides whether and how to disclose the vulnerabilities it discovers or acquires. No other country with major cyber capabilities, such as China, Russia, and Iran, is known to have a similar process. In many cases, then, Mythos may well be discovering vulnerabilities in U.S. critical software infrastructure that U.S. adversaries had already found, and may already have been silently exploiting.

An even more interesting wrinkle is that Mythos may help private corporations like Microsoft discover vulnerabilities that the United States government has found and chosen not to disclose to them. We know for certain that the U.S. government has historically known about vulnerabilities in widely used consumer and enterprise software owned by American corporations and not disclosed those vulnerabilities to the company in question. Thus, diffusion of Mythos to U.S. software companies—especially those with the largest customer bases—may increase the security of global consumers and businesses while decreasing, on the margin, the cyberoffensive capabilities of the U.S. government. In the abstract, this is a trade I’d happily take.

To be clear, there are other arguments against overreacting to Mythos. Perhaps you think Anthropic is “hyping” its dangerous capabilities for “marketing” (I disagree with you, but fair enough). If that is what you believe, your argument will not stand the test of time, and indeed in my view it is already outdated.

A related argument is that OpenAI released GPT 5.5, which in some dimensions seems to have similar performance to Mythos. It seems as though Mythos remains meaningfully ahead in open-ended cyber vulnerability discovery, however. Given that the U.S. government is asking Anthropic not to release Mythos beyond the highly limited group of firms that have received it, while they have not objected to OpenAI’s general public release of GPT 5.5, it seems the policy planners in the federal government agree with the assessment that Mythos is importantly superior to 5.5 in at least some ways.

Regardless of the status of the horse race as of this instant, the nature of AI progress is such that we can rest assured that OpenAI and its US competitors will train models that meet or surpass what Mythos can do today soon. Even if “GPT 5.5 being public shows that Mythos isn’t as dangerous as Anthropic says,” it is clear that argument will not last very long. Thus, this second argument has the same flaw as the first argument; if it is not already outdated (which I think it is), it will become outdated soon.

Or perhaps, more reasonably, you argue that most cyberattacks today are accomplished using social engineering strategies (e.g. phishing) that AI systems far less capable than Mythos have already been enabling (for example, being able to write convincing, contextually appropriate emails to thousands of phishing targets simultaneously). This latter argument is not wrong, but it should not be taken to mean “vulnerabilities don’t matter.” Attackers have relied on social engineering techniques because of the expense, time, and difficulty associated with vulnerability discovery. If AI changes that calculus—as I have argued Mythos does—it means that both vulnerability-based attacks and social-engineering-based attacks become much easier. This is an argument for defending against both kinds of cyberattacks, not an argument about ignoring one at the expense of the other.

Mythos-and-beyond capabilities can ultimately be a boon for cybersecurity. But this outcome requires a wide range of cyberdefenders, large and small, to use it to find and fix their security holes. That is likelier to happen if capabilities of this type are broadly diffused as opposed to kept hidden within governments.

Instead of these alternative arguments—which rely either on betting against deep learning getting better (a bad bet) or on a kind of willful myopia—the argument I have tried to paint is instead the structural argument against overreaction to Mythos. It will be true regardless of how good AI cyber capabilities get, at least so long as there remains a distinction between private corporations and the state. Yet my argument is not that there is no role for the state. Instead my argument would suggest the need for a structured, publicly legible, and clearly bounded role for the state.

So what should we do about Mythos, concretely?

—

There are three questions that any governmental response to Mythos must address:

1. Should models of “Mythos” capabilities be released to the general public when their training is complete, as is the case for models today? Is that a safe thing to do? Or would there ideally be a delay between the attainment of new, potentially dangerous capabilities and their broad public availability? This is a prudential question. If political leaders decide that the answer to this is no, then a series of technocratic questions emerge: on what basis do we decide that a model is “too dangerous to release?” How long do we withhold public release? What would make a model go from being “too dangerous to release” to being “safe enough”? What do we do with the models while their availability is restricted, and who is authorized to do those things?

2. How will we disseminate information about the vulnerabilities we find to a)the companies that own or maintain the software in question and b)the users of that software, who may be wholly decentralized, i.e. not easily able to receive software security updates from the company responsible for maintaining the software?

3. Given the expectation that AI cyber capabilities will continue to improve, neither (1) nor (2) are long-term sustainable solutions. Instead, the future of software security will involve fundamental changes to the way we approach the problem. How will we identify those changes and ensure their rapid adoption throughout the economy?

I am going to give very cursory answers to questions two and three because I would like for question one to be my focus. The answer to question two is probably something like “a public-private consortium that coordinates between various government-established tiers of critical infrastructure providers.” Project Glasswing itself is a good example of such a consortium; formalizing and expanding it to both other frontier labs and other critical infrastructure providers (especially owners of physical-world infrastructure) would be, broadly speaking, the right move.

The answer to question three is probably “fund scientific research organizations, including focused-research organizations (FROs), to work on formal verification and other methods of provably secure software development, and once these have reached high readiness levels, disseminate them through the economy using the consortium described in the answer to question two as well as leveraging the standard and obvious market incentives that would exist to adopt something like ‘provably secure software.’”

Now back to question one.

First, the prudential question: should models that advance the cyber vulnerability discovery capabilities frontier—like Mythos—be made publicly available as soon as they are finished being trained? Most people now seem to agree the answer to this is “no.” I’ll take that as a starting point.

The current, nascent process by which government appears to be conducting pre-deployment screening models is highly informal and clunky. For example, it seems that Anthropic asked the White House for permission to expose Mythos to a somewhat larger set of critical infrastructure providers, and the White House said no. There is no formal deliberative process the government used to make this decision, and the decision rests on no particular legal authority. “Just ‘cause we said so” is basically the frontier AI governance policy of the United States today.

This is basically a licensing regime, but one with no basis in either the law or in technical reality. The Trump Administration stumbled into this dynamic, so to be clear I am not arguing any of this is the result of deliberate action. But we should be clear: the current trajectory of federal frontier AI governance is worse than the direction of AI policy under the Biden administration. And it is much worse than the trajectory implied by the catastrophic-risk transparency regimes of New York and California.

It would be tragic if the accelerationists, when given power in the Trump Administration, ended up being so unprepared to perform AI risk management that they stumbled into a regulatory regime worse than the one devised under President Biden, which they found so draconian.

The way to avoid this outcome in the near term (the next few months) is for the Center for AI Standards and Innovation (CAISI) within the Commerce Department (the new name for the AI Safety Institute), the Department of Energy, and the intelligence community to develop evaluations that define thresholds for unacceptable risk within the narrow domain of cyber vulnerability discovery. The best-available open-weight model anywhere in the world should set the absolute floor for this threshold and be adjusted upward every time an open-weight model advances the open-weight capabilities frontier.

The tests could be performed on the “helpful-only” version of the model (with no guardrails of any kind); the version with model-level, but no system-level, safeguards; and the version with both model- and system-level safeguards, with the model’s final score a weighted or non-weighted average of all three tests. This allows the test to capture some element of the risk of model jailbreaks and other malicious circumventions of developer safeguards. If a U.S. company developing either open- or closed-weight models can advance consumer and enterprise-relevant capabilities without advancing cyber vulnerability discovery, or some other risk category to be defined in the future, they can release their model without having to stagger the release.

There are fundamental problems with this approach. The scale and range of the risk categories will grow tremendously in the coming years. The diversity and complexity of safeguards will rise. All of these things will have to be measured, evaluated, and graded in rapid time.

There are serious questions about whether CAISI can handle these responsibilities on its own. Even the current challenges may outstrip the capabilities of any U.S. government agency, including CAISI. CAISI is not officially authorized by Congress and there are cabinet secretaries whose Washington pied-a-terres have higher valuations than CAISI’s budget of less than $10 million. The White House just fired the newly hired director of CAISI because they didn’t like that he came from Anthropic—rich, given that the entire reason we are having this conversation is because of a model Anthropic developed. One might imagine a member of the technical staff that built Mythos would have some useful insights to offer, but those insights were collateral damage in a pointless political fight between Anthropic and the government. I want CAISI to succeed as much as the next person, but it is simply untenable to rely solely on them for such a crucial role in the government response.

There is also a question about whether CAISI should have this responsibility in the long term. First, CAISI is part of the National Institute for Standards and Technology (NIST)—a non-regulatory federal agency. Everything described above is clearly regulatory. It is essentially against the cultural DNA of NIST to assume such a regulatory role, and there is considerable reason to believe CAISI would be hindered by internal politics relating to this.

Finally there is the question of whether the government should solely have this responsibility. There are the practical questions: does it have the money and the technical capacity (e.g., the compute) and the personnel and the political fortitude required to do a good job at the incredibly complex work I have described? I think the answer is no. Then there are the structural questions. Recall my argument in the prior section about the structural incentive of government in particular to control frontier AI to do violence and lessen the security of American individuals and businesses in favor of “national security.” This structural tendency does not go away with time—it gets worse.

A government regulator with unilateral power to restrict model releases will have a massive incentive to hoard models from the public, thereby ensuring that the American and global economies do not benefit nearly as much from frontier AI as they otherwise could, while only unevenly increasing worldwide security. Such a regulator also exposes frontier AI to immense amounts of political interference. If you are a Democrat, imagine what Donald Trump and Pete Hegseth would do with that level of power. If you are a Republican, imagine what Kamala Harris or Joe Biden or Gavin Newsom would do with it.

You can be motivated by the practical questions about whether government should unilaterally take on the role I am describing, by the structural questions, or by both (I am motivated by both). But either set of them, I think, is dispositive: the government is going to need significant help from private institutions if it is to manage the AI transition successfully. And to prevent political interference, we will need mediating institutions between raw public force and purely private corporate property.

This line of thought is what led me to my work on private governance—publicly overseen and licensed, privately managed, nonpartisan bodies whose responsibilities would include verifying the safety claims and procedures of the frontier AI companies on behalf of the government and the public at large. This ecosystem is still maturing, but it has grown meaningfully from where it was one year ago. I am fully persuaded that if an authorizing legal framework for such bodies to operate were to exist, significant money, talent, and computing power would be made available to these groups by non-conflicted philanthropic parties.

Such an authorizing framework, combined with the catastrophic-risk transparency framework already passed by California and New York (described in the introduction), would make for an excellent federal AI law to preempt a patchwork of state regulations. One could therefore kill two birds—one of them an accelerationist concern and the other an AI safety concern—with one stone. In the absence of a federal framework, state laws to authorize Independent Verification Organizations—which would function more or less as I have described the private bodies above—are under consideration in state legislatures (full disclosure: I am an affiliate of Fathom, a non-profit group that has worked on such legislation, though I have played no role in drafting, legislative strategy, or lobbying for those bills).

—

I want to emphasize that everything I have described is frontier AI governance during the transition from a pre-AI world to a post-AI world. This is a very narrow thing. It is a critically important thing, but it is more like the parachute a crew capsule uses to slow itself down as it descends from space than it is like “all the rules that should obtain once the humans exit the crew capsule and build a society on a newly colonized planet.” If you read this and still have questions about “how all this will work in the long run,” welcome to the party.

Among pilots, there is a mantra for what to do in an emergency: “aviate, navigate, communicate.” It means that controlling the aircraft safely takes priority over “knowing where we are headed,” which takes priority over explaining the situation to anybody else. We are still in the “aviate” stage, and that means you should expect many things to remain ambiguous, incomplete, or confusing. This is policymaking in triage mode, and we are likely to remain there for some time to come.