N NVIDIA Last verified

NVIDIA GB200 NVL72

NVL In production Released 2024 blackwell-gen1
BF16
TFLOP/s
2250 厂商声称
FP8
TFLOP/s
4500 厂商声称
FP4
TFLOP/s
9000 厂商声称
Memory
GB
192 厂商声称
Mem BW
GB/s
8000 厂商声称
TDP
W
1200 厂商声称

Full specs

Compute

FP4 TFLOPS
9000
FP8 TFLOPS
4500
BF16 TFLOPS
2250
FP16 TFLOPS
2250
INT8 TOPS
4500

Memory

Capacity
192 GB
Bandwidth
8000 GB/s
Type
HBM3e

Die architecture 🟢 vendor floorplan

SM count
160
Tensor cores / SM
4
L2 cache
100 MB
HBM stacks
8
Process
4 nm

Scale-Up (intra-node)

Protocol
NVLink-5.0
Per-link BW
1800 GB/s
World size
72
Topology
switched-rail-optimized
Switch
nvswitch-gen4

Scale-Out (inter-node)

Per-card NIC
800 Gbps
Protocol
InfiniBand-XDR
NIC
ConnectX-8

Topology

拓扑结构 · Topology
72 卡 scale-up domain
芯片内部 / Die-level architecture
HBM HBM HBM HBM HBM HBM HBM HBM NVIDIA GB200 NVL72 L2 / shared cache · NoC L1$ / register file (per SM) 160 SMs · darker block = tensor / matrix engine 2250 TFLOPS BF16 · 4500 FP8 · 192 GB HBM3e @ 8.0 TB/s · 1200 W TDP

🟢 vendor floorplan 160 SMs · 8× HBM · 100 MB L2 · 4 nm

集群拓扑 / Cluster topology · NVLink-5.0 @ 1800 GB/s
Spine (NVLink-5.0 fabric) Leaf switches N1 N2 N3 N4 N5 N6 N7 N8 N9 Super-pod (rack-scale) · 72 cards in single scale-up domain · 1800 GB/s/link · 2-tier Clos fabric
Scale-Up · 域内
NVLink-5.0
1800 GB/s · 拓扑: switched-rail-optimized
world_size = 72
Scale-Out · 跨域
InfiniBand-XDR
800 Gbps/卡 NIC
ConnectX-8

Which models can it run?

Quick estimates · decode tok/s/card 上界

TP=8 · FP4 · batch=16 · prefill=1024 · decode=256 · 已应用 efficiency 校准

在计算器中调整 →
模型 参数 (active) Decode tok/s/card 瓶颈
DeepSeek V4 Pro
deepseek
 49B 163,265 内存带宽
DeepSeek V4 Flash
deepseek
 13B 227 内存带宽
Mistral Small 4
mistral
 22B 104 内存带宽
GLM-5 Reasoning
zhipu
 32B 86 内存带宽
GLM-5.1
zhipu
 32B 58 内存带宽
Qwen3.6 Plus
alibaba
 35B 56 内存带宽
Kimi K2.6
moonshot
 32B 48 内存带宽
MiniMax M2.7
minimax
 46B 38 内存带宽

Operator-level fit · per-model bottleneck + upper bound

算子级 fit · operator-level fit (per-token roofline)

基于每个模型 operator_decomposition + 本卡 BF16 2,250 TFLOPS / 8,000 GB/s 计算 · ridge point ≈ 281 FLOPs/byte

上界 = min(计算屋顶, 内存带宽屋顶) · efficiency 未应用
模型 domain 主导算子 AI · F/B 瓶颈 tok/s 上界
DeepSeek V4 Pro llm matmul 245.5 💾 内存带宽 327k
GraphCast scientific graph-message-passing 0.9 💾 内存带宽 15k
AlphaFold 3 scientific pair-bias-attention 2.3 💾 内存带宽 4435
GPT-OSS llm matmul 0.7 💾 内存带宽 647
Gemma 4 26B llm matmul 0.7 💾 内存带宽 481
DeepSeek V4 Flash llm matmul 0.8 💾 内存带宽 455
Mistral Small 4 llm matmul 0.6 💾 内存带宽 207
Llama 4 Maverick llm matmul 0.8 💾 内存带宽 205
需要 efficiency 校准 + concurrency 扫描 + TCO 估算 → 在计算器中评估 →

Operator support & optimization headroom

算子支持 & 优化空间 / Operator support & headroom

Per-operator support derived from software_support.engines + scale-up topology. Optimization headroom from measured efficiency factor.

Optimization headroom
+50 pp
moderate

No cases yet — using default 0.5 efficiency. Real headroom unknown until first measurement lands.

Communication (collective)
All-to-All 🟢 mature
all-to-all via NVLink-5.0 world_size=72
AllReduce 🟢 mature
NVLink-5.0 ring all-reduce
Attention
Multi-Head Attention 🟢 mature
paged-attention via vLLM/SGLang/MindIE
FlashAttention-3 🟢 mature
FA-3 on modern engine + tensor cores
Matrix multiply (GEMM)
Matrix Multiplication 🟢 mature
GEMM supported on all inference engines
MoE routing
MoE Routing 🟢 mature
MoE gating supported via vLLM ≥0.4 / SGLang
Normalization
RMSNorm 🟢 mature
fused into engine kernels
Embedding
Rotary Position Embedding 🟢 mature
fused into engine kernels
Activation
SiLU / Swish 🟢 mature
fused into engine kernels
Softmax 🟢 mature
fused into engine kernels

最接近的替代卡 (按规格相似度)

基于 BF16 算力 / 显存 / 显存带宽 / FP8 加权欧氏距离。供选型决策参考。

NVIDIA B200 SXM 180GB
NVIDIA
100%
相似度
BF16
2250
Mem
180
BW
8.0
AMD Instinct MI300X
AMD
99%
相似度
BF16
1307
Mem
192
BW
5.3
Etched Sohu
Etched
98%
相似度
BF16
1125
Mem
144
BW
5.8
NVIDIA H200 SXM 141GB
NVIDIA
98%
相似度
BF16
989
Mem
141
BW
4.8

Software-stack support

Engine Status BF16FP16FP4FP8 E4M3FP8 E5M2INT4 AWQ
HanGuangAI unconfirmed — — — — — —
LMDeploy unconfirmed — — — — — —
MindIE unconfirmed — — — — — —
MoRI official ✓ — ✓ ✓ — —
SGLang official ✓ — ✓ ✓ — —
TensorRT-LLM (Dynamo) official ✓ — ✓ ✓ — —
vLLM official ✓ — ✓ ✓ — —

Existing deployment cases (0)

No measured cases yet for this card. Be the first contributor?

Citations

  1. [1] NVIDIA GB200 NVL72 product page; per-GPU specs (each GB200 contains 2 B200s) — https://www.nvidia.com/en-us/data-center/gb200-nvl72/ · accessed 2026-04-28 厂商声称
  2. [2] GB200 reuses Blackwell B200 die: 160 SMs, 100 MB L2, 8× HBM3e (per-GPU); 2 GPUs + 1 Grace CPU per Bianca board — https://resources.nvidia.com/en-us-blackwell-architecture · accessed 2026-04-28 厂商声称
⚠ All performance figures are vendor-claimed unless tier=measured.
⚠ GB200 chip pairs 2 B200 GPUs with 1 Grace CPU; specs above are per-B200.