0

How you plan for this constraint could determine the fate of your AI architecture

In sum – what to know:

Inflection point is close — The thermal wall hits when air cooling reaches its physical limit at 40kW.

HVAC to fluid dynamics — Liquid cooling is going from optional to unavoidable.

Skills shortage pervades — Hire fluid dynamics engineers sooner rather than later.

Guy Massey’s “cooling roadmap” and accompanying analysis are getting a lot of attention, and as a director of global service delivery for CommScope, he knows what high-density environments look like. By showing cooling innovation as the inflection point, he offers a glimpse into when frameworks will have to adapt so that cooling can shift from airflow to fluid dynamics.

In Massey’s image, the inflection happens somewhere around 40kW, when server components (CPUs and GPUs) generate enough heat that traditional air cooling is no longer effectively or efficiently dissipating the heat without significant trade-offs in energy use and performance.

When businesses start to truly embrace AI for real-world problems, with strategic integration of AI into core business processes and workflows, training and analytics workloads will push operators toward 50kW-120kW, with more CPUs and GPUs packed into racks. The power density per rack (kW/rack) will become an increasingly important factor for design and capacity planning, with some facilities on track to exceed 60 kW/rack (though the norm today remains around 8-12 kW/rack).

Though the average data center operator is far from the extreme 340kW densities of HPE, Dell, Vertiv, Flex, and Nvidia, there is, no doubt a rapid increase in rack density across the board, primarily driven by the demands of AI, ML, and HPC. This is reshaping data center infrastructure.

To stay ahead of the curve, data center operators will have to shift from HVAC to fluid-dynamics, like direct-to-chip and immersion cooling (especially for more demanding workloads that warrant the significant upfront Capex). They will also have to keep an eye on advances in thermal management as computational power grows over the next few years. Advances to watch would be:

• Leveraging the boiling of dielectric fluids on chip surfaces, moving from single-phase cooling to 2-phase liquid-to-vapor cooling for 2x to 3x the heat removal;
• Directly circulating refrigerant to processors for heat absorption, with refrigerant-to-air systems and direct-to-chip systems that could eliminate the need for water;
• Leveraging non-water dielectric refrigerants for direct-to-chip cooling, mitigating the risk of shorts or damage when in direct contact with CPUs, GPUs, and motherboards;
• And by 2030, wider-spread adoption of the Nordic countries’ practice of heating homes and businesses with hot water pumped from data centers to nearby buildings.

Each of these evolutionary stages will help operators manage the intense heat and tackle the challenges of uneven cooling, high energy and water usage, and integration of emerging and legacy technologies, but only if they hire the right people at the right time.

Hire skilled people sooner rather than later

With competiton from booming sectors like aerospace/defense and manufacturing, it’s important to tap talent early on. Now is the time when data center operators should be thinking about the skill sets and how to blend HVAC techs and fluid dynamics engineers. In addition, they have to make sure those engineers have a command of coolant chemistry, manifold design, and pressure control, as well as the collaboration skills to work with IT, mechanical, and engineering teams.

According to LVI Associates, which recruits talent within the energy and infrastructure sectors, a specialized workforce will be composed of:

• Mechanical and process engineers, who design fluid circuits and pressure-balanced systems.
• Electrical and automation engineers to manage sensors, pumps, and control systems.
• Facilities technicians that handle service tasks, inspect flow rates, and maintain coolant integrity.
• Health, safety, and environmental specialists to oversee fluid handling and recycling to meet sustainability standards.

Other areas of investment to consider include data analytics, predictive maintenance, IoT (sensors), and automation tools that will help data center teams monitor and respond to changes in critical parameters like temperature, vibration, power usage, and humidity. The investment will enable a shift from reactive to proactive operations, leading to  improved overall efficiency and reliability, and possible cost savings.