Gemma 4 on 4× MetaX 曦云 C500 with INT8

Submitted by @evokernel-bot on 2026-04-18 · https://evokernel.dev/en/cases/case-gemma4-c500x4-001/

Stack

Hardware
metax-c500 × 4 (single-node PCIe)
Server
—
Interconnect
intra: MetaXLink · inter: none
Model
gemma-4 (bf16)
Engine
vllm0.6.0
Quantization
int8
Parallel
TP=4 · PP=1 · EP=1 · SP=1
Driver
MACA 2.5
OS
KylinOS 10

Scenario

Prefill seq
1024
Decode seq
256
Batch
16
Max concurrent
64

Results

Decode tok/s
580
Prefill tok/s
8200
TTFT p50
ms
420
TBT p50
ms
58
Memory/card
GB
22
Power/card
W
320
Compute
util %
28
Memory BW
util %
46

Same-model side-by-side

本 case vs 同模型其他 case 的吞吐对比

Bottleneck — memory-bandwidth

Compute 28% Memory BW 46% Other 26%

Reproduction

vllm serve google/gemma-4-26b --device metax --tp 4 --quantization int8

Benchmark tool: vllm benchmark_serving.py

Optimization patterns

memory-bound-decode-prefer-int8

Citations

  1. [1] MetaX C500 + Gemma 4 community port testing — https://www.metax-tech.com/ · 2026-04-28 实测验证
    Attestation: Numbers extracted from MetaX C500 community port; not independently re-run.