Ahead-looking: A brand new report has revealed the big quantity of Nvidia GPUs utilized by Microsoft and the improvements it took in arranging them to assist OpenAI practice ChatGPT. The information comes as Microsoft pronounces a major improve to its AI supercomputer to additional its homegrown generative AI initiative.
In keeping with Bloomberg, OpenAI skilled ChatGPT on a supercomputer Microsoft constructed from tens of 1000’s of Nvidia A100 GPUs. Microsoft introduced a brand new array using Nvidia’s newer H100 GPUs this week.
The problem going through the businesses began in 2019 after Microsoft invested $1 billion into OpenAI whereas agreeing to construct an AI supercomputer for the startup. Nevertheless, Microsoft did not have the {hardware} in-house for what OpenAI wanted.
After buying Nvidia’s chips, Microsoft needed to rethink the way it organized such a large variety of GPUs to forestall overheating and energy outages. The corporate will not say exactly how a lot the endeavor price, however government vp Scott Guthrie put the quantity above a number of hundred million {dollars}.
Additionally learn: Has Nvidia gained the AI coaching market?
Concurrently working all of the A100s compelled Redmond to contemplate the way it positioned them and their energy provides. It additionally needed to develop new software program to extend effectivity, make sure the networking gear may stand up to large quantities of knowledge, design new cable trays that it may manufacture independently, and use a number of cooling strategies. Relying on the altering local weather, the cooling methods included evaporation, swamp coolers, and out of doors air.
Because the preliminary success of ChatGPT, Microsoft and a few of its rivals have began work on parallel AI fashions for search engines like google and yahoo and different functions. To hurry up its generative AI, the corporate has launched the ND H100 v5 VM, a digital machine that may use between eight and 1000’s of Nvidia H100 GPUs.
The H100s join by means of NVSwitch and NVLink 4.0 with 3.6TB/s of bisectional bandwidth between every of the 8 native GPUs inside every digital machine. Every GPU boasts 400 Gb/s of bandwidth by means of Nvidia Quantum-2 CX7 InfiniBand and 64GB/s PCIe5 connections. Every digital machine manages 3.2Tb/s by means of a non-blocking fat-tree community. Microsoft’s new system additionally options 4th-generation Intel Xeon processors and 16-channel 4800 MHz DDR5 RAM.
Microsoft plans to make use of the ND H100 v5 VM for its new AI-powered Bing search engine, Edge net browser, and Microsoft Dynamics 365. The digital machine is now accessible for preview and can come customary with the Azure portfolio. Potential customers can request entry.
