Table of Contents
Microsoft launched its first AI-focused chip, the Azure Maia accelerator, and the Cobalt CPU in 2023, and is working on next-generation versions
In sum – what to know:
Microsoft plans chip self-reliance – CTO Kevin Scott said the firm ultimately wants its own silicon to power AI data centers, reducing reliance on Nvidia and AMD.
End-to-end system design focus – Microsoft aims to control chips, networks, and cooling to optimize compute for workloads, with new in-house chips and microfluid cooling underway.
Compute crunch intensifies – Despite billions in new capacity, Microsoft still warns of persistent shortages as AI demand outpaces its forecasts.
Microsoft aims to rely primarily on its own silicon for data center AI workloads in the long run, the firm’s CTO Kevin Scott said.
Speaking at Italian Tech Week, moderated by CNBC, Scott noted that Microsoft currently uses a mix of Nvidia, AMD, and in-house chips, but emphasized that the aim is to control more of the system design — from processors to cooling — to better match workloads and expand compute capacity.
The company launched its first AI-focused chip, the Azure Maia accelerator, and the Cobalt CPU in 2023, and is working on next-generation versions.
During the event, the executive stressed that demand continues to exceed supply. “A massive crunch is probably an understatement,” he said, adding that since ChatGPT’s launch, even the firm’s most ambitious forecasts have fallen short. Despite significant new data center buildouts, Microsoft expects capacity needs to keep rising sharply.
Last week, Microsoft announced that it had successfully developed an in-chip microfluidic cooling system that can effectively cool a server running core services for a simulated Teams meeting.
“Microfluidics would allow for more power-dense designs that will enable more features that customers care about and give better performance in a smaller amount of space,” said Judy Priest, corporate vice president and chief technical officer of cloud operations and Innovation at Microsoft.
“But we needed to prove the technology and the design worked, and then the very next thing I wanted to do was test reliability,” Priest said.
The company’s lab-scale tests showed microfluidics performed up to three times better than cold plates at removing heat, depending on workloads and configurations involved. Microfluidics also reduced the maximum temperature rise of the silicon inside a GPU by 65%, though this will vary by the type of chip. The Microsoft team expects the advanced cooling technology would also improve power usage effectiveness, a key metric for measuring how energy efficient a datacenter is, and reduce operational costs.
