DeepSeek V4 Flash on 16× MTT S4000 (Moore Threads KUAE)

Submitted by @evokernel-bot on 2026-04-23 · https://evokernel.dev/en/cases/case-dsv4flash-mtts4000x16-001/

Stack

Hardware
mtt-s4000 × 16 (2 nodes × 8 cards)
Server
moore-threads-kuae
Interconnect
intra: mtlink · inter: roce-v2
Model
deepseek-v4-flash (bf16)
Engine
vllm0.6.0
Quantization
fp16
Parallel
TP=8 · PP=2 · EP=1 · SP=1
Driver
MUSA 3.5
OS
KylinOS 10

Scenario

Prefill seq
1024
Decode seq
256
Batch
16
Max concurrent
64

Results

Decode tok/s
320
Prefill tok/s
5800
TTFT p50
ms
540
TBT p50
ms
78
Memory/card
GB
38
Power/card
W
410
Compute
util %
22
Memory BW
util %
56

Same-model side-by-side

本 case vs 同模型其他 case 的吞吐对比

Bottleneck — software

Compute 22% Memory BW 56% Other 22%

Reproduction

vllm serve --device musa --tp 8 --pipeline-parallel-size 2 deepseek-ai/DeepSeek-V4-Flash

Benchmark tool: vllm benchmark_serving.py

Issues encountered

  • MUSA 3.5 vLLM 移植版尚未支持 FP8; 退化到 FP16
  • EP > 1 时性能反而下降 (路由通信成本太高)

Optimization patterns

moe-expert-routing-on-domestic

Citations

  1. [1] Moore Threads KUAE community benchmark sharing — https://www.mthreads.com/ · 2026-04-28 实测验证
    Attestation: Numbers extracted from Moore Threads community port testing; not independently re-run.