This submit was co-authored by Hugo Affaticati, Technical Program Supervisor, Microsoft Azure HPC + AI, and Jon Shelley, Principal TPM Supervisor, Microsoft Azure HPC + AI.
Pure language processing (NLP), automated speech recognition (ASR), and text-to-speech (TTS) functions have gotten more and more widespread in immediately’s world. Most firms have leveraged these applied sciences to create chatbots for managing buyer questions and complaints, streamlining operations, and eradicating a few of the heavy value burden that comes with headcount. However what you might not understand is that they’re additionally getting used internally to cut back danger and establish fraudulent conduct, scale back buyer complaints, enhance automation, and analyze buyer sentiment. It’s prevalent in most locations, however particularly in industries resembling healthcare, finance, retail, and telecommunications.
NVIDIA lately launched the most recent model of the NVIDIA NeMo Megatron framework, which is now in open beta. This framework can be utilized to construct and deploy massive language fashions (LLMs) with pure language understanding (NLU).
Combining NVIDIA NeMo Megatron with our Azure AI infrastructure gives a robust platform that anybody can spin up in minutes with out having to incur the prices and burden of managing their very own on-premises infrastructure. And naturally, we’ve got taken our benchmarking of the brand new framework to a brand new stage, to actually present the facility of the Azure infrastructure.
Reaching new milestones with 530B parameters
We used Azure NDm A100 v4-series digital machines to run the GPT-3 mannequin’s new NVIDIA NeMo Megatron framework and take a look at the boundaries of this collection. NDm A100 v4 digital machines are Azure’s flagship GPU choices for AI and deep studying powered by NVIDIA A100 80GB Tensor Core GPUs. These cases have essentially the most GPU reminiscence capability and bandwidth, backed by NVIDIA InfiniBand HDR connections to assist scaling up and out. Finally, we ran a 530B-parameter benchmark on 175 digital machines, leading to a coaching time per step of as little as 55.7 seconds (figure1). This benchmark measures the compute effectivity and the way it scales by measuring the time taken per step to coach the mannequin after regular state is reached, with a mini-batch measurement of 1. Such excellent velocity wouldn’t have been attainable with out InfiniBand HDR offering glorious communication between nodes with out elevated latency.
These outcomes spotlight an virtually linear velocity enhance, guaranteeing higher efficiency for the next variety of nodes—paramount for heavy or time-sensitive workloads. As proven by these runs with billions of parameters, prospects can relaxation assured that Azure’s infrastructure can deal with even essentially the most tough and sophisticated workloads, on demand.
“Velocity and scale are each key to creating massive language fashions, and the most recent launch of the NVIDIA NeMo Megatron framework introduces new methods to ship 30 p.c sooner coaching for LLMs,” mentioned Paresh Kharya, senior director of accelerated computing at NVIDIA. “Microsoft’s testing with NeMo Megatron 530B additionally exhibits that Azure NDm A100 v4 cases powered by NVIDIA A100 Tensor Core GPUs and NVIDIA InfiniBand networking present a compelling choice for reaching linear coaching speedups at huge scale.”
Showcasing Azure AI capabilities—now and sooner or later
Azure’s dedication is to make AI and HPC accessible to everybody. It contains, however just isn’t restricted to, offering the most effective AI infrastructure that scales from the smallest use instances to the heaviest workloads. As we proceed to innovate to construct the most effective platform to your AI workloads, our promise to you is to make use of the most recent benchmarks to check our AI capabilities. These outcomes assist drive our personal innovation and showcase that there isn’t any restrict to what you are able to do. For all of your AI computing wants, Azure has you coated.
Be taught extra
To study extra concerning the outcomes or the best way to recreate them, please see the next hyperlinks.
