FP8 E4M3
fp8 有损
4-bit exponent, 3-bit mantissa; preferred for activations due to dynamic range
权重位数
bits/weight
8
激活位数
bits/activation
8
支持硬件
of total
22/39
实测案例
6
支持硬件 (22)
国产 (1)
海外
AMD Instinct MI300A AMD Instinct MI300X AMD Instinct MI325X AMD Instinct MI355X AWS Trainium 2 Cerebras WSE-3 Etched Sohu Google TPU Trillium (v6e) Groq LPU (TSP v1) Intel Gaudi 2 Intel Gaudi 3 NVIDIA B200 SXM 180GB NVIDIA B300 SXM 288GB NVIDIA GB200 NVL72 NVIDIA GB300 NVL72 NVIDIA H100 SXM5 80GB NVIDIA H200 SXM 141GB NVIDIA L40S NVIDIA R200 SXM (Vera Rubin) SambaNova SN40L Tenstorrent Wormhole n300
使用此量化的案例 (6)
- DeepSeek V4 Flash on 8×H100 SXM with vLLM FP8h100-sxm5 ×8 · deepseek-v4-flash · 4200 tok/s
- Qwen2.5-Coder 32B on 4× L40S with vLLM (FP8)l40s ×4 · qwen2.5-coder-32b · 580 tok/s
- DeepSeek V4 Flash with disaggregated prefill (H100) + decode (H200) via Mooncakeh200-sxm ×16 · deepseek-v4-flash · 9600 tok/s
- Qwen3.6 Plus on 8× MI325X with SGLang FP8mi325x ×8 · qwen3.6-plus · 3100 tok/s
- Gemma 4 26B on 4× H100 SXM with FP8h100-sxm5 ×4 · gemma-4 · 6800 tok/s
- GPT-OSS on 8× Intel Gaudi 3 with vLLMgaudi-3 ×8 · gpt-oss · 2900 tok/s