On Recursive Self-Improvement (Part I)

Thoughts on the automation of AI research

Introduction

America’s major frontier AI labs will have begun automating large fractions of their research and engineering operations. The pace of this automation will grow during the course of 2026, and within a year or two the effective “workforces” of each frontier lab will grow from the single-digit thousands to tens of thousands, and then hundreds of thousands.

This means that soon, the vast majority of frontier AI lab staff will neither sleep nor eat nor use the bathroom. They will grow smarter and more capable each month, not only because AI itself was already improving quickly but because the only objective these hundred-thousand-strong workforces will have is to make themselves smarter.

The automation of AI research and engineering is probably the most important thing that will happen in the field of AI over the coming year (and one of the most important things in the history of the field), but it is frustrating to talk about because it is unlikely to ‘happen’ in one recognizably discrete event, and indeed in some important sense it is already happening. More frustrating still is the fact that it will take place almost entirely behind closed doors.

Make no mistake: AI agents that build the next versions of themselves—is not “science fiction.” It is an explicit and public milestone on the roadmaps of every frontier AI lab. OpenAI has been the most transparent: they envision hundreds of thousands of automated research “interns” within about nine months from today, and a fully automated workforce in about two years.

There is substantial uncertainty about what, exactly, automated AI research will mean. It could simply mean that AI capabilities progress unfolds faster, but within the familiar “generative AI” paradigm. This might not matter as much as many in the AI industry believe. It could also mean fundamental changes to the nature of AI itself and to the strategic dynamics that obtain over the field; these are the concerns that animate the AI 2027 project. To be clear, though, the debate should really not be whether this automation will occur but instead about how it will occur—the details and the implications.

Policymakers would be wise to take especially careful notice of this issue over the coming year or so. But they should also keep the hysterics to a minimum: yes, this really is a thing from science fiction that is happening before our eyes, but that does not mean we should behave theatrically, as an actor in a movie might. Instead, the challenge now is to deal with the legitimately sci-fi issues we face using the comparatively dull idioms of technocratic policymaking.

This week, I’ll walk you through the range of scenarios I think may be possible with automated AI research and how it may affect the dynamics of AI development. Next week, in Part II, I will argue that regardless of the outcome, the automation of AI research and development changes the fundamental dynamics of the field enough to merit targeted policy action.

In both pieces, there is one assumption I’ll ask you to make with me, which is that substantial automation of AI research is a near-term possibility. This requires believing a few things. First, that AI research and engineering is substantively composed of work like: finding optimizations in various complex software systems; designing and testing experiments for AI model training and posttraining; and creating software interfaces to expose AI model capabilities to users. Second, that a great deal of this work is essentially reducible to the engineering of software. Third, that AI models, while not yet geniuses, are reaching quite high levels of human competence. Fourth, that frontier lab leadership and staff are serious when they describe AI research automation as a near-term goal, and that frontier lab research staff are telling the truth when they say that AI is already writing a large fraction of their code.

These all strike me as reasonable propositions, and I’ll ask you that you join me in the assumption that these propositions taken together mean that “automated AI research of some form or another will happen soon.” This will allow us to explore the interesting questions about automated AI research, as opposed to asking whether it will happen at all, which would require devoting thousands of words to rehashing debates about AI capabilities that were interesting in 2024 and have grown less interesting to me by the month. These debates have largely been settled by empirical reality, and it is long past time to move on and accept the enormity of what is unfolding at the frontiers of AI.

What Might Automated AI Research and Engineering Be Like?

Imagine yourself standing by a street and seeing a Bugatti race by you at 200 miles per hour; a few minutes later, a second Bugatti speeds by at 300 miles per hour. This difference in velocity is huge to anyone inside the car, and any seasoned observer of motorsport would know that a mere 200 miles per hour is common Bugatti territory, whereas 300 miles per hour is approaching a world record for conventional vehicle speed. But the random bystander on the street might not notice much of a difference between “extremely fast” and “world historically fast.”

The current rate of AI capabilities improvement has already surpassed the ability of most humans to keep track. It is therefore entirely possible that the automation of AI research may lead to a dramatic acceleration in AI capabilities advances, and that most of the public (and policymakers) will not really notice, especially in the early stages of the automation. The predictable result of this will be that pundits say, “those AI hypesters promised us that this supposed ‘AI research automation’ would finally mean that AI would live up to its promises; but once again, it has just resulted in more of the same empty promises.” Therefore, this scenario—call it the “AI as a normal exponentially self-improving technology”—is the bearish scenario for AI research automation.

But something else is possible, too. Imagine that instead of merely traveling 100 miles per hour faster, the second Bugatti learned how to fly. The bystander on the street would notice the flying Bugatti not so much for its speed but for the fact that it is flying. And imagine even further that it really was the Bugatti itself that learned how to fly; the humans ostensibly at the steering wheel can explain to the public what the Bugatti did to make itself fly, but it was not ultimately their work. The Bugatti built them a joystick that operates the vehicle in flight, but again, the joystick was built by the machine rather than the humans. No human personally wired it all together, and while there exist detailed specifications for every single component, they too were not written by humans.

As our street bystander ponders this incredible feat and tries to sort his way through the millions of word written by the Bugatti explaining how it achieved flight, the Bugatti lets him know that it just figured out a way to reduce its price by 99%. When this was a mere car, it was one of the most expensive in the world. Now it flies and costs as much as a Toyota Corolla. Oh, and the Bugatti also informs the bystander that it is pursuing a new engineering path that could allow the car to leap 30,000 feet into the air in a matter of seconds, as well as operate underwater. This is more speculative, but the car reckons it can make meaningful progress within a year or so.

This is obviously a heavily stylized metaphor, but you get the idea. In one outcome, the automation of AI engineering is hugely important yet doesn’t result in a fundamental change to the dynamics of the field. In the other, something altogether new is afoot.

Automated AI Research Within the Labs

We don’t know which of these scenarios describes our future more accurately. Differences in intuition about this question explain many downstream disagreements on near-future AI capabilities and how much AI needs to be regulated.

Those with a more bearish view on AI research automation point out that diminishing returns are common in the field. At the risk of torturing the Bugatti metaphor to death: as a car picks up speed, the amount of energy required to continue accelerating increases nonlinearly. It requires about twice as much energy to accelerate from 200 miles per hour to 250 miles per hour than it does to accelerate from 100 miles per hour to 150, even though the absolute quantity of acceleration (50 miles per hour) is identical.

The same dynamic often obtains in artificial intelligence; famously, the ‘scaling laws’ that appear to describe the relationship between computing power, data, and model performance suggest that order of magnitude increases in input resources yield an additional ‘nine’ of reliability. In other words: for ten times more compute, you can go from 9 to 90 percent, but then the next tenfold increase in compute only brings you from 90% to 99%, and then 99% to 99.9%, and so on. Very quickly, astronomical amounts of compute are needed for only miniscule improvements to capability.

Those who are more bullish on AI research automation retort by observing that the field of AI is still replete with low-hanging fruit well beyond naïve scaling of resources along the lines described above. In particular, they point to an extremely broad set of improvements known as ‘algorithmic efficiency.’ Dario Amodei has said that, with human-driven research and engineering, individual labs achieve something like 400% efficiency improvements per year. Amodei describes this as a “compute multiplier”: the same amount of compute can deliver a model that is 4 times better than would otherwise be possible without the efficiency improvements.

These gains come from all sorts of places: model architecture tweaks that improve how well the model learns from training data or leverage compute more effectively; improvements to training datasets that allow the model to learn more quickly; enhancements to the tooling and technical infrastructure that labs use to train and deploy models; and many other things besides. Collectively, small and medium-sized gains add up to the 400% efficiency improvements Amodei describes. AI research and engineering, in practice, is the grinding pursuit of these little gains far more often than it is the pie-in-the-sky investigation of entirely new paradigms (though to be clear, this does happen within labs too). Indeed, lab executives have said repeatedly and for years that the distinction between research (high status, rarefied) and engineering (the low-status grind) is false, and that in practice the two disciplines often converge. This is why OpenAI, famously, uses the job title “Member of the Technical Staff,” rather than “researcher” and “engineer.”

We know that AI labs are bound by talent; this is why they routinely offer top-tier personnel compensation packages in the tens or hundreds of millions of dollars. For the sake of illustration, imagine that a lab has 1000 research and engineering staff, 800 of whom are grinding away in the search for gains within the current paradigm, and 200 of whom are investigating new paradigms. Both do their jobs by designing and conducting experiments in an iterative fashion; in both a huge amount of this work can be described as “writing code” and “engineering software.” They run the experiments, they analyze the results, and they write up their findings.

This is of course an absurdly high-level description, but it is also not that hard to imagine current models automating large portions of this work. And indeed, it seems clear that they already do; frontier lab staff now frequently say that AI models write most or all their code. Current models are arguably already better at coding than many human researchers (particularly when considering the opportunity cost of the researchers’ time), and the direction of travel on this trend is obvious (I bet you the models will get better!).

Where the models currently suffer from reliability and quality issues is in the execution of experiments over say, several days, though they improve along these dimensions constantly. The other deficit of the models—and here they often fall down altogether—is in the generation of interesting hypotheses and research agendas. A brilliant human researcher may have some insight about a new direction of research and spend months refining his thesis, recruiting colleagues to his cause, and persuading management to allocate compute to test his ideas. Models do not tend to come up with great insights like this.

But do they need to? What if instead, the brilliant human researcher had an army of automated junior researchers he could use to test his ideas in a way that would have been impossible without advanced AI? Models could autonomously perform countless iterations on the human’s fundamental experimental insight. It is hard to imagine a world in which this capability does not end up being utterly transformative to the work of the researcher, but there are still questions one can ask about the broader impact. If, for example, compute remains a binding constraint on labs, then the allocation of compute to different research directions will still be a matter of bureaucratic process, and ultimately, politicking within frontier labs. These are messy human processes; they require hashing out differences of opinion and making fundamental strategic tradeoffs about what kinds of research to pursue. To say the least, this seems much more difficult to automate than coding.

Thus one can imagine two things happening in parallel. First, the 800 researchers who are grinding away within the current paradigm suddenly have vastly more bandwidth to search for more efficiency gains. An extreme outcome from this would be that labs discover dramatically more efficiency gains; it turns out there was a vast field of low-hanging fruit just waiting to be plucked, and we simply did not have enough researcher time to find it. In this world, perhaps algorithmic efficiency gains scale cleanly with the number of automated researchers: 10 times the number of researchers means that annual improvements in algorithmic efficiency are 4000% rather than 400%.

But this seems unlikely. Maybe the human researchers were doing a pretty good job all along, and discovering most of the algorithmic efficiency gains that were practical to employ. In this world, perhaps automated researchers merely double the rate of improvement (800% per year) or even worse, accelerate it by, say, 20% (for a final annual efficiency gain of 480% per year). In the most extreme bear scenario, there are literally no new gains to be discovered, so all automated researchers do is enable labs to find the same gains they would have counterfactually found with humans, but much faster than with purely human-driven research.

Then, at the same time, those 200 “new paradigm” researchers suddenly have the ability to systematically investigate novel research directions with far greater velocity than is currently possible. Determining how much this matters requires considerable speculation; how good are the researchers’ ideas? How many novel directions are there left to pursue in deep learning, and how many of them can be pursued without the collection of new, real-world datasets? For example, perhaps we can train robots capable of automating all physical labor with enough human perception data, but the collection of that data cannot be automated by AI researchers. One can be pessimistic that this will mean radical acceleration of research progress, but it would be strange to imagine no meaningful gains in the progress of new research paradigms.

Add to this the reality that all frontier labs will bring massive new computing resources online within the coming year. These data centers are dramatically larger than anything that has come before, and are really the first manifestation of the AI infrastructure boom. Remember for example that we have not seen any models trained on Blackwell-generation chips, and soon each lab will have hundreds of thousands each of them (and the Rubin generation will begin real-world deployment in the coming months, too). For all the talk of AI infrastructure, we really have not seen what our AI industry can go with gigawatt-scale computing power. There is a credible case to be made that there is a looming capabilities overhang from this alone, and that this overhang will be realized at the same time that automated AI researchers begin to be deployed in earnest. It would therefore be wise to expect 2026 to be a more rapid year of AI progress than all years that have come before. Indeed, this is probably the conservative forecast at this point.

All of this together creates a clear picture: This year, the automation of AI research and engineering will begin in earnest. In addition to creating at least a step-change improvement in AI progress from its already rapid pace, this could change the dynamics of AI competition, alter AI geopolitics, and much more. Next week, I will discuss a targeted policy measure to shed more light on this development as it unfolds.