Nvidia PODs: shrinking time-to-first-token'

Delays in data center construction means delays in the time it takes to generate AI tokens. Nvidia hopes to provide a stopgap for delayed brick-and-mortar construction with its new DSX PODs, which it introduced today at its GTC event in Berlin.

The units – a single Vera Rubin DSX POD – include 7 liquid-cooled compute racks, which are interwoven with 11 miles (17 kilometers) of Spectrum-X cabling, and NVLink switching. Power is to be ushered between racks and hot liquid cooling to be managed by DSX MaxLPS and DSX OSS.

Nvidia claims that the shipping-container sized unit can drop deployment timelines from an average of 6 months down to 14 days, greatly expediting token-generating capacity in the time it takes to erect brick-and-mortar buildings.

Its primary goals are to maximize “tokens per watt” and to recover stranded data center capacity, with DSX PODs providing a reference design for closed-loop liquid cooling and dry coolers, which Nvidia contends could bring potential savings of 54M liters/day (for a 2-GW deployment).

Additionally, the DSX MaxLPS software helps operators use up to 30% less power to generate the same number of tokens, with early adopters including CoreWeave, Firmus, Google Cloud IREN, Lambda, Nebius, Nscale and Sharon AI.

Nvidia isn’t the first to come up with the “modular” idea, as Crusoe, Dell, HPE, Lenovo, Vertiv, and others have rolled out pre-integrated “pods,” but Nvidia’s version is more of an ecosystem reference blueprint for standardizing partner-built layouts than a proprietary “container,” per se. That allows infrastructure players like Schneider Electric or Vertiv to build units that conform to the DSX blueprint, but using their own CDUs, power skids, and physical containers.