Table of Contents
Three layers define the AI boom, and each moves at its own speed
In sum – what to know:
Adapt for growth and change – Data center operators are adapting to handle more than just growth, but also ‘big, fast’ change
Navigating speed and tension – Three layers of AI infrastructure are moving at different speeds, opening “gaps” in speed-to-power and capacity
Hardware as the catalyst – Where software used to ‘eat the world,’ it’s hardware that is driving the speed of AI infrastructure buildouts
Physical infrastructure simply cannot be built fast enough to keep up with AI’s insatiable, ever-growing appetite. In today’s RCRTV AI TechTalk, Richard Miller, founder of Data Center Frontier and creator-in-chief of Data Center Richness, discusses his “Data Centers 2026: Solving The Speed Squeeze,” which focuses on three layers of the AI boom, and the different speeds at which each layer is evolving:
- Data centers and AI factories
- Software and GPUs
- Utilities and the Power Grid
The speed mismatch
“We are in a period of incredible growth but also what I call ‘big, fast change,’ which is not what the data center industry is really used to – growth, yes, but the speed at which things have to change, including some of the ways we build and operate data centers, is what’s really different with the current AI boom,” says Miller. “That creates significant tension in how speed issues, particularly speed-to-power for data centers, gets reconciled and how the industry moves forward.”
To Miller, Nvidia’s hardware has been the primary catalyst for the current AI revolution, with GPUs powerful enough to handle complex algorithms and to enable autonomous agents and models that handle agentic AI and real-time inference — all of which were not possible with previous chip architectures. “That layer is moving really quickly, with new models all the time, and companies like OpenAI, Anthropic and Google racing one another to constantly iterate and to put out better and more useful models.”
At the same time models are evolving, Nvidia is accelerating the pace at which it puts out more powerful GPU chips and software, as evidenced by its recent VeraRubin architecture. “Where equipment and servers used to refresh every 3-to-5 years, the hardware is now moving at more of an annual cadence.” This shift, Miller says, requires specialized, “custom-built” facilities designed for extreme power density and liquid cooling to manage the rapid obsolescence of, and high demands from, new hardware.
“Where the data center layer for many years supported Netflix, YouTube, health records, online banking, and videoconferencing during Covid, AI folks want things yesterday,” which Miller says forces a fundamental paradigm shift in how digital infrastructure must be built. “When you’re plowing new hardware into data centers…folks look for more capacity and that’s where the power layer comes in, but the utility industry and power grid moves at a different speed.” This is the most critical part of the argument about a speed mismatch, as new chips can be designed and shipped within a year, whereas grid speed is much longer.
“There’s a reason for the disconnects between the speed at which these layers are moving. The old saying is that ‘software is eating the world‘ and doing it quickly, updates quickly, and moves quickly. We’re not quite used to the hardware moving as quickly as Nvidia has pushed the pace. And with data centers, the best case traditionally was 18 to 36 months as a pretty good timetable for building a data center, but now, the wait for utility power from local utilities and larger power grid in major data center markets can be 3 to 7 years.”
As the speed of GPUs and new models greatly increases infrastructure demands, data centers and AI factories are increasingly needed to handle the extreme heat and massive amounts of electricity required by new GPUs. Miller’s contention is that it’s no longer feasible to “plug in” to old facilities, but rather necessary to shift toward AI factories with custom-engineered liquid cooling and high-density power.
“With VeraRubin, the power that the servers use to run the calculations to do the data crunching, and for cooling, will be exponentially higher than it is now,” he says, explaining that current Nvidia gear is running at about 120-130 kW/rack, with VeraRubin expected to get up to 600 kW/rack.
For that reason, liquid cooling is now the backbone of new data center builds. “It’s non-trivial for data centers. It’s more expensive, but you can theoretically make more revenue on the back end with that.” In other words, the plumbing – pipes, pumps, and coolant distribution units – that must integrate directly into server racks will be a major engineering challenge (i.e., for cold plates, immersion tanks, leak prevention), but worth it in the long run. Despite the higher up-front CapEx, liquid is 3,000x more effective than air at carrying heat away.
Because of the amount of power these units require, Miller says there’s a larger rethink of power distribution within data centers. “A rethink of how the power gets from the grid to the rack, so that higher power voltage comes to the chip. With liquid cooling ,rather than the heat coming off of the chips and being blown through, you instead of have the cooling come right to the chip, with liquid going through the cold plate above the processor, which is the more efficient way to take the heat away,” says Miller, acknowledging some parts of the server and chassis, like the memory, still may need some air cooling.
The issue is that the volume and voltage of power that arrives at the rack: “That means the whole system, the UPS, the transformers that step down power from the grid, because you are dealing with a much larger volume of power on the towers that come to the data center, as opposed to what arrives at the rack to run the processor. All that infrastructure is being rethought and redesigned.”
Miller believes data center cooling has been revolutionized in its evolution from evaporative cooling to where we are today with closed loop systems. “Water scarcity is pushing the need for closed loop cooling, where rather than constantly using water and running it through a cooling tower, you charge it once, and then that water is re-circulated so it’s heated and cooled within that system.” Of note is the fact that Miller believes the public is grossly misinformed on this issue. “Right now, that is poorly understood in the public discussion, the new data centers, like from Microsoft in its Fairwater data center design, uses exponentially less water.”
He contends the industry “has not done the best job in describing how the technology has changed.” The culture of not disclosing water use has been passed forward from the “evaporative water” days, which is no longer relevant today. “A lot of the reporting in the media and advocacy by community groups has been largely based on numbers from evaporative cooling in areas that were not experiencing water scarcity. That’s very different from new AI data centers today, where they are largely using closed loop techniques. It’s important for a community to ask ‘what does the cooling look like,’ to get an accurate picture. If they’re somewhere with water scarcity, then they have to push for innovations.”
$630 billion from ‘Big 4’
In terms of how the three layers of AI infrastructure will drive the $630 billion projected to be spent by the “biggest hyperscalers, Miller says it’s a 62% jump from the record $388 billion spent in 2025. The focus is land, power, and custom buildings to house the next generation of chips. But how much is really immediate GPU deployment versus “land banking”?
“2025 was the most audaciously unprecedented amount of data center construction we’d ever seen; it blew the doors of anything historically,” admits Miller, but he feels a lot of the $630 billion will go toward “reserving forward capacity for GPUs and for powered land and other investments in the data center facilities themselves.” He asks, “how much new investment can this industry actually absorb, and how much will actually address issues in 2026?” He feels the capex will be doled out a little bit at a time, entering the economy bit by bit.
Inference versus training
In current earnings, data center and AI operators see a big increase in inference, including from enterprise users “that are starting to use AI tools that are getting better, and as people get used to them, inference is growing,” says Miller. He points to three different schools of thought around inference: “Large training facilities that are training growth models can be used for inference as well.” He says the networks are getting good enough that from a latency perspective, they will be adequate, “especially for thinking models that take a minute or two to do something like generate an infographic, or spreadsheet, or a podcast.”
He says for some data center engineers, if all training comes to a screeching halt, they can run inference. But yet others might say, “no, we need capacity close to the users…around the major markets, where the use will be concentrated.” For that reason, Miller says the major metros will have smaller modular data centers delivering inference inference and AI servers to new places. “Not at a 100 MW scale, but rather smaller like 5-10 MW facilities.”
Yet others think phones are getting powerful enough, and models small enough, that inference could be on the device. Miller says inference on the device could be efficient, particularly for some of the open source models, but for others, it might be a combination of all of the above. “The infrastructure will go where the need is.”
Build-your-own power
Bridging the gap between speed-to-power and how to get the capacity will continue to be the biggest challenge in 2026, with the build-your-own power movement being the biggest trend Miller sees for 2026. “If power isn’t coming from the utility, then data center and AI operators are going to build their own. That’s the biggest trend this year. Operators and their financial partners are getting into the power business in a bigger way – building on-site power often using natural gas generation that gets them up and running faster than if they wait for the utility connection,” says Miller, noting that in some cases, bridging capacity is a big concern that is driving the need to rent generators and run power locally, until the utility connection is possible. He points to Elon Musk’s xAI as an example of on-site and adjacent power as a bridge.
Despite some of the criticism this has engendered in local communities, as with Musk’s 41 natural gas burning turbines in Mississippi, Miller believes renewables will be one of the more exciting pieces of the puzzle by 2035.
Renewables: firm vs. intermittent power
“Some of the bigger hyperscalers want to be 100% renewable,” says Miller, eluding to projects by Google, Microsoft, Meta, and Amazon that focus on nuclear, solar, wind, geothermal, and other carbon- and water “negative” goals. All the companies’ leaders have talked about wanting to remove more carbon than they emit, and some have talked about goals to be “water positive” in the next few years. “There is and should be a push for climate goals… to shift to renewables, and to accelerate,” says Miller. “Getting renewable capacity on the grid, the power that is deployed at largest scale and most cheaply is solar.” The trade-off, he notes, is that solar, wind, and other renewables require a lot of land and resources when it comes to data center scale. “The intermittent supply and the need for firm power [that is available 24/7] is driving a greater focus on energy storage and renewables that offer 24/7 options, like geothermal.
When it comes to grid dynamics, and how to balance the need for firm power 24/7 versus intermittent, Miller says it’s “getting cheaper and easier to scale to data center levels,” with companies like Google committing to solar and storage innovations. “These will be the exciting projects going forward.”
He is referring to Google’s 24/7 carbon-free energy (CFE) goal, with a 2030 timeline. The company is transitioning from a strategy of simply purchasing renewable energy certificates to directly investing and partnering on new solar, wind, nuclear, geothermal and long-duration storage projects.
Amazon, Meta, and Microsoft are operating at a similar or even larger scale in terms of total clean energy procurement, with the top four accounting for nearly half of all global corporate clean power deals in 2025, although some predict there will be a sharp drop off in 2026, because of regulatory rollbacks (like the Big Beautiful Bill, which rolled back clean-energy tax credits), grid saturation, and a pivot toward firmer power like nuclear and natural gas.
