Introduction
Securely Accelerate Workloads
From Enterprise to Exascale
H100 features fourth-generation Tensor Cores and a Transformer Engine with FP8 precision that provides up to 4X faster training over the prior generation for GPT-3 (175B) models.
For high-performance computing (HPC) applications, H100 riples the floating-point operations per second (FLOPS) of double-precision Tensor Cores, delivering 60 teraflops of FP64 computing for HPC while also featuring dynamic programming (DPX) instructions to deliver up to 7X higher performance.
With second-generation Multi-Instance GPU (MIG), built-in NVIDIA Confidential Computing, and NVIDIA NVLink Switch System, H100 securely accelerates all workloads for every data center, from enterprise to exascale.
Features
Key Benefits of NVIDIA H100 TensorCore GPUs
NVIDIA Ampere Architecture
Built with 80 billion transistors using a cutting-edge TSMC 4N process custom tailored for NVIDIA’s accelerated compute needs, H100 features major advances to accelerate AI, HPC, memory bandwidth, interconnect, and communication at data center scale
Transformer Engine
The Transformer Engine uses software and Hopper Tensor Core technology designed to accelerate training for models built from the world’s most important AI model building block, the transformer. Hopper Tensor Cores can apply mixed FP8 and FP16 precisions to dramatically accelerate AI calculations for transformers
NVLink Switch System
The NVLink Switch System enables the scaling of multi- GPU input/output (IO) across multiple servers at 900 gigabytes per second (GB/s) bidirectional per GPU, over 7X the bandwidth of PCIe Gen5. The system delivers 9X higher bandwidth than InfiniBand HDR on the NVIDIA Ampere architecture
NVIDIA Confidential Computing
NVIDIA H100 brings high-performance security to workloads with confidentiality and integrity. Confidential Computing delivers hardware-based protection for data and applications in use
Second-Generation Multi-Instance GPU (MIG)
The Hopper architecture’s second-generation MIG supports multi-tenant, multi-user configurations in virtualized environments, securely partitioning the GPU into isolated, right-size instances to maximize quality of service (QoS) for 7X more secured tenants
DPX Instructions
Hopper’s DPX instructions accelerate dynamic programming algorithms by 40X compared to CPUs and 7X compared to NVIDIA Ampere architecture GPUs. This leads to dramatically faster times in disease diagnosis, real-time routing optimizations, and graph analytics
Artificial Intelligence
Supercharge
Large Language Model Inference
With H100 NVL
For LLMs up to 70 billion parameters (Llama 2 70B), the PCIe-based NVIDIA H100 NVL with NVLink bridge utilizes Transformer Engine, NVLink, and 188GB HBM3 memory to provide optimum performance and easy scaling across any data center, bringing LLMs to the mainstream.
Servers equipped with H100 NVL GPUs increase Llama 2 70B model performance up to 5X over NVIDIA A100 systems while maintaining low latency in power-constrained data center environments
Conclusion
Experience GPU-Accelerated
Cloud Computing
Our mCloud platform, built on robust OpenStack architecture, now offers the ability to integrate powerful NVIDIA GPUs directly into your virtual machines through GPU passthrough technology. This allows virtual machines to access the full capabilities of a physical GPU as if it were directly attached to the system, bypassing the hypervisor’s emulation layer and providing near-native performance.
Enhance your cloud capabilities with GPU acceleration by integrating dedicated GPUs into your mCloud virtual machines and take your computing to the next level.
See How Much You Can Save with mCloud
Customize your cloud and compare costs instantly against AWS, Google Cloud, and Microsoft Azure. Get more for less with enterprise-grade performance.
- Transparent Pricing: No hidden fees or surprises.
- Enterprise-Grade for Less: High performance at lower costs.
- Instant Comparison: See real-time savings.
