Microsoft recently released its latest ND H100v5 virtual machine series, which is equipped with cutting-edge NVIDIA H100 Tensor Core GPU and NVIDIA Quantum-2 InfiniBand networking technology. Designed to handle the most complex AI and machine learning tasks, these virtual machines provide unprecedented computing power and efficiency.
At the core of the ND H100v5 series is its AI supercomputing GPU, each virtual machine is equipped with eight NVIDIA H100 Tensor Core GPUs, which are connected to the host via the PCIe Gen5 interface, and each GPU has a bandwidth of up to 64GB/s, greatly optimizing the data transmission efficiency of the host-to-GPU.
In addition, the series also uses the fourth generation Intel Xeon Scalable processor, ensuring maximum processing speed and computing power. Combined with DDR5 memory, the ND H100v5 virtual machine performs well when handling large-scale datasets, providing huge memory bandwidth and capacity.
In terms of networks, the application of NVIDIA Quantum-2 ConnectX-7 InfiniBand technology ensures low latency and high throughput for data transmission, which is crucial for AI applications that need to process large amounts of data quickly.
In terms of performance, the ND H100v5 virtual machine is six times faster when using the new 8-bit FP8 floating-point data type for matrix multiplication, faster than the previous generation of FP16. This improvement is particularly significant when dealing with large language models such as the BLOOM175B end-to-end model, achieving up to twice the speed increase in reasoning.
This innovation by Microsoft not only demonstrates its leadership in the fields of cloud computing and AI technology, but also provides enterprises and research institutions with powerful tools to promote the development and application of AI technology. With the continuous advancement of AI technology, the ND H100v5 virtual machine series will undoubtedly become an important cornerstone of future AI supercomputing.