Recently RCRTech interviewed HPC and AI infrastructure pioneer David Driggers, who, as CEO and founder of Cirrascale, has architected specialized, bare-metal server solutions explicitly optimized for heavy multi-GPU deep learning workloads and heavy-duty AI training. He has also begun a major pivot toward enterprise-focused dedicated inferencing and inference-as-a-service for Fortune 500 companies, which we speak about in today’s AI TechTalk and featured article.

Notably, he says a one-size-fits-all approach is impossible from an accelerator perspective, explaining that “as we move to a mixture of experts and multimodal type inferencing where you may be integrating audio, video, plus text, and ultimately spatial, different accelerators will excel at different things.” For that reason, it’ll be very important in inferencing for enterprises to find the right platform for different needs, whether that’s for ultra-low latency, energy efficiency, lowest possible cost per token, or other requirements. “You should seek the smallest, simplest unit your model will fits into, and then push it down the technology stack as far as you can go…while still meeting your latency requirements – your time to first token.” That he says is essential to keep costs low, as “every semiconductor company charges more, the higher you move up their technology stack, charging per flop and per megabyte of memory.” 

To get more insights, read the article here, and view the video here.

Adaptive reuse for DCs: JLL’s Sean Farney tells RCR Tech that paper mills, steel plants and manufacturing facilities are increasingly being converted into data centers, particularly across the U.S. Rust Belt, where power is already established.