Table of Contents
Can an assembly-line approach to high-density AI compute-in-a-box present a componentized answer to inefficient brick-and-mortar data centers?
In sum – what to know:
High-density AI cubes — Each component offers ~0.6 MW of compute (600-700 kw per cube), with clusters of 4 – 6 MW.
Componentized structure — a hybrid system with a schematic of a manifold of direct-to-chip cooling and a structure of single row or double row cubes, with heat dissipated between the rows.
Reduce time-to-compute — Cut months or even years from data center projects, with 20% overall savings.
Some believe the AI race will be over in less time than the average transmission line project takes to reach operation in the US (10-15 years). To bend the cost-per-MW curve during the data center construction boom, there should be a shift toward iterative and repeatable designs that rapidly scale up to meet time-to-compute demands.
One inventor who specializes in mobile distributed energy resources and cooling modules for data centers is thinking out of the box, or better yet, in the cube, about how to shift data center design away from physical, site-constructed design to an assembly-line approach that enables rapid production and deployment of data centers.
“The U.S. must industrialize AI infrastructure at a speed that rivals wartime efforts, with data centers built the way aircraft, automobiles, and batteries are built — on assembly lines as opposed to solid, bespoke construction sites,” said Chad Caldwell, founder and CEO of Swarm Infrastructure, which manufactures decentralized cooling and decentralized UPS systems designed to unlock traditional architectures from the constraints of brick-and-mortar envelopes and inefficient cooling. “The U.S. AI advantage will lie not only in the AI hardware stack, like advanced semiconductors, but also in the balance-of-system (BOS) required to power and cool them. That BOS innovation will be crucial to maintaining a competitive edge against China’s manufacturing capabilities.” Because actual inventory takes up only 30-40% of the space in a traditional data center, the rest of the space is used to move that inventory around. “That means data center operators are paying to move air around,” said Caldwell.
For that reason, Caldwell has spent the past few years building modular 8 ½ x 9 ½ ‘ high decentralized cubes that can create hundreds of quasi-independent data centers working together as one larger, more efficient data center. Each cube delivers high-density AI compute, with ~0.6 MW of compute (600-700 kw per cube), and clusters offering 4 to 6 MW. Data center operators can add and subtract cubes according to what is needed, with all connecting to the same power bus.
Each cube contains:
- Decentralized two-phase direct-to-chip cooling for GPUs like Nvidia Rubin/Vera
- Decentralized vapor-compression air cooling for secondary thermal loads
- Liquid-cooled busbars
- Integrated 400-800VDC battery topology for compute flexibility
- Centralized rectifiers/inverters
- Controls that balance compute, cooling, power and grid integration
Operators can stack 4 servers across, 4 cubes high, for approximately 3 to 4 stories of height. As an example, an Edge data center would use one truck of 6 cubes, and a 1 GW data center would use 2,000 cubes. “This is where the crucial element of ‘flexibility’ for servers comes in, as you drive the truck to the site, plug it in, and you’re up and running. With a common power bus, you wrap the cubes together,” said Caldwell, who contends there can be a 20% decrease in costs with each doubling of output.
Like a ‘giant Yeti cooler’
To get a better idea of how it works, think of a high-performance, rotomolded cooler like a Yeti cooler, but bigger, and in multiple rows, dissipating heat between the rows. Each Swarm cube is encased in an “envelope” of 3-inch rotomolded cooler-type material, made through biaxial rotomolding, with a cooling loop that goes directly to atmosphere air. It’s a hybrid system with a schematic of a manifold of direct-to-chip cooling. “When you look at two phase cooling it’s essentially refrigeration—phase change vapor compression cooling, but a little bit different in that the chips are working as the evaporators, and you put the condenser on the roof [rather than a heat exchanger], right out into atmosphere,” explained Caldwell.
In other words, instead of having a UPS and power conversion inside of a physical data center, you build a componentized structure of single row or double row cubes, with heat dissipated between the rows. “You can do wall or roof panels, but it’s an insulated layer, with as much as possible moved outside of the insulated layer so that the batteries that produce heat and have power conversion are outside,” said Caldwell.
By shrinking the “envelope,” so to speak, Caldwell hopes data center operators will be able to dedicate all the space to their servers, with space in-between to circulate the air through very large evaporator coils, with direct-to-chip accounting for 75%-85% of the heat load, and then a 6’ axial fan (or four 3’ axial fans) to push the air for the secondary heat loads through the server racks, down through the evaporator, and back up to take care of the secondary heat.
This would be a change in concept, as today, data center operators use 2’ diameter water pipes to get heat out of a building. Instead of using massive amounts of water to expel heat from a building, Caldwell believes the direct-to-chip with a cooling loop going directly to the atmosphere air is a superior construct, not only because of the more efficient cooling, but also the impending obsolescence that will be built into data centers. “Servers will depreciate in 5 years, so why put that into a building and wait for a gas power or a turbine, which takes years to get?”
He likens the approach to what Tesla is doing. “Tesla delivers pre-cast, all electrical chargers on a truck, and they set it up and take off. That’s what our concept is.” To learn more about the liquid cooling, air cooling, and computational fluid dynamics of Swarm’s solution, contact Caldwell on LinkedIn.
