← 量化方案

INT4 AWQ

awq 有损

Activation-aware Weight Quantization; weight-only int4

权重位数
bits/weight
4
激活位数
bits/activation
16
支持硬件
of total
9/39
实测案例
0

支持硬件 (9)

国产 (1)

昇腾 950

海外

AMD Instinct MI300X AMD Instinct MI325X Apple M4 Max Neural Engine NVIDIA A100 SXM4 80GB NVIDIA B200 SXM 180GB NVIDIA H100 SXM5 80GB NVIDIA H200 SXM 141GB NVIDIA L40S

使用此量化的案例 (0)

尚无案例引用此量化方案。