Out of Thin Air

A proposal for the grid

Welcome back to Hyperdimensional. Many of you are new subscribers, so I’d like to step back for a moment and tell you what you should expect as a subscriber.

Hyperdimensional is a weekly (sometimes a bit more) newsletter about AI, emerging technology, and the future of governance. Topics include analysis of state and federal AI policies proposed by others, my own policy proposals (some technocratic, others quite broad or speculative), my thoughts about the current state and future direction of AI (including “reviews” of recent AI releases), and the occasional bigger picture piece about the role of technology in society and culture.

My time in government has substantially broadened the range of policy topics about which I have things to say. For example, in the earlier phase of Hyperdimensional, I rarely engaged on issues like AI in healthcare or finance, or how to ensure policy accommodates the construction of AI infrastructure. I now have quite a bit to say about such things. From time to time, I will depart from my traditional Hyperdimensional style to explain precisely how I suggest operationalizing various aspects of America’s AI Action Plan. These essays will be less romantic and discursive than the style to which longtime readers will be accustomed. Today’s essay is an example of this. Essays closer to my usual style will continue to be the Hyperdimensional norm.

I will always try to honestly describe the world as I see it and make good-faith recommendations for how policymakers should respond. At the same time, my thoughts are constantly evolving, and everything I say is provisional, a snapshot in time. Not only do I expect my thoughts and recommendations to change over time; I actively hope for it.

It is good to be back with you, writing on a weekly basis. With that, let’s dive in.

-

The United States should investigate new and novel ways for large power consumers to manage their power consumption during critical grid periods to enhance reliability and unlock additional power on the system.
America’s AI Action Plan

One of the few things AI policy observers can agree to in their forecasts of the near future is that AI will require tremendous electricity to continue scaling. Supplying this electricity will undoubtedly require vast new infrastructure, including natural gas plants, nuclear fission plants, and, eventually, enhanced geothermal and nuclear fusion. Building this infrastructure, in turn, demands smart deregulation in environmental and energy law. This is a story, and a policy playbook, that will be familiar to most people who have read about AI’s power demands.

Yet executing this playbook well is insufficient. First, even the most basic electricity infrastructure—transmission lines, transformers, natural gas turbines, and the like—are badly supply constrained. These bottlenecks are unlikely to be resolved fully in the next few years. Second, there is substantial engineering, financial, and logistical execution risk associated with nuclear fission, nuclear fusion, and enhanced geothermal. Even if the deregulatory agenda goes perfectly, it does little to mitigate those more basic risks.

So America will need more than just deregulation. Fortunately, there is low-hanging fruit to be plucked. America can pull gigawatts out of thin air through a combination of technology and smartly targeted policy. Let me show you how.

Using Demand Response to Manufacture Gigawatts

It is often said that the US electricity grid is under increasing risk of blackouts, price spikes, and other signs of strain. This is true, but this claim belies another reality: the grid is also oversupplied most of the time. The grid is built to sustain electricity service during periods of peak demand—the hottest summer day in Texas, or the coldest winter in New England. A combination of extreme weather events and increased electrification of industrial and residential power uses means those periods of peak demand place more acute stress on the grid.

Most of the time, however, the grid has significantly more power than is needed. This means that the grid can often accommodate, say, a new 750-megawatt data center for the vast majority of the year. There is spare generation capacity available except for those brief periods of high demand. But in the high-demand periods, when all the grid’s generation capacity is required to maintain electricity service, this new data center would require an additional 750 megawatts of electricity generation capacity, and quite possibly also transmission infrastructure upgrades.

This additional investment is only necessary if you assume that the new data center will require all 750 megawatts of electricity during peak-demand periods. Traditionally, this assumption has been true: data center operators rely on extremely high uptime, and grid operators work under the assumption that new electricity demand will be constant during periods of high demand.

If, however, that assumption were not true, and a data center was able to significantly reduce or eliminate its electricity consumption for a small portion of the year (the high-demand period), the calculus changes radically. More power would suddenly become available because the data center can tap into the grid’s existing surplus capacity without requiring investment in net-new capacity on the days when the grid is operating at the limits of its capacity.

How much more power could be unlocked? In a viral paper earlier this year, Tyler Norris and colleagues at Duke University estimated 76 gigawatts if the new users of that power were willing to curtail their electricity demand for 0.25% of the year. In overly simplified terms, this means that America could accommodate 76 gigawatts of new AI data centers today, with no new power generation built, if those data centers were willing to reduce their demand by an average equivalent of roughly 22 hours out of a year.

As it happens, the estimates I trust most about near-term AI-related electricity demand suggest that we will need about 50-75 gigawatts for AI over the coming 5 years—perfectly in line with Norris’ estimates. The amount of power unlocked becomes even larger with higher curtailment times (e.g., Norris estimates 98 gigawatts could be unlocked with just a 0.5% average annual curtailment rate—less than two days out of the year).

In addition to instantly unlocking more power for AI and other industrial applications, curtailing power at the scale envisioned in the Duke study would achieve other benefits. For example, as Norris observes, more efficient use by industrial customers of existing power generation capacity during non-peak demand periods would result in high utilization rates of existing capital assets, and thus lower prices for consumers.

The result is a win-win for both AI data center operators and average Americans concerned about the affordability and reliability of electricity. The only downside would be that, during periods of peak demand (for example, on a particularly hot day in one region of the country), AI users across America might notice their AI services being slower and less reliable than usual. This seems well worth the cost.

How, then, do we achieve this outcome? The idea of electricity consumers reducing their consumption during high-demand periods is known as “demand response,” and it has been around for a long time. You may have received a text from your electric utility this summer asking you to turn your air conditioning down; this is a primitive example of demand response. Some utilities offer customers (whether residential or industrial) price discounts in exchange for participating in more sophisticated demand response programs, typically where the utility can unilaterally lower your demand (for example, through a smart thermostat).

While data centers have historically been reluctant to participate in demand-response programs, there are ample technological means for them to do so. Such programs need not entail binary choices: data centers can reduce their electricity use without shutting it off entirely. In fact, dynamically reducing the power consumption of thousands of GPUs operating in parallel in response to a signal from the grid (“reduce power by 50% in the next ten minutes”) might strike you as a problem that deep learning itself can be helpful in solving. And indeed, there are startups doing just this.

The central problem is that as of today, most data centers do not receive sufficient benefits for participating in demand response programs. Marginal electricity costs are a small fraction of the overall cost of operating an AI data center, so the electricity price benefits offered by utilities today are of little value for data centers in particular.

What is it that today’s AI data center developers and operators crave the most? Time to power. When a new data center (or any other industrial facility) goes online, they must request an interconnection to the grid from their local utility. The utility then models the effect of that new demand on the grid. If the result of that modeling is a prediction that the facility will require new power generation or transmission infrastructure, the owner of the new facility must pay the costs of construction, while the construction itself is managed by the utility.

The timelines for all of this (and note: this is a very rough, highly simplified sketch) can be years. Part of the reason this process works the way it does is that, as a general matter, utilities typically assume that the power draw of a new facility will remain firm even in periods of peak demand, unless that new facility explicitly agrees otherwise.

Thus, the obvious trade comes into view: offer data center operators accelerated interconnection (and thus faster time to power) in exchange for participating in demand response programs and employing the software and hardware necessary to be able to provably reduce power consumption at a few minutes’ notice.

Would this solve all problems with powering AI? Absolutely not. It will still be crucial to expand the overall capacity of the grid overall, and in some regions this trade will be less feasible than others (because some regions have more overhead capacity at off-peak times than others). But this trade would meaningfully unlock new capacity for AI and other power-hungry industrial facilities while giving America more time to build new energy infrastructure.

Some utilities are beginning to offer faster interconnection in exchange for participation in demand response programs, but the federal government can formalize, standardize, and accelerate this for utilities throughout the country. A draft Executive Order that operationalizes this is available here. I offer it for demonstration purposes only for those interested in the technical details. Of course, one need not necessarily operationalize this in an Executive Order; as ever, one can simply just do things.