操作系统及版本
oe2203.aarch64
安装工具的python环境
docker容器中的python环境
python版本
3.10
AISBench工具版本
v3.1-20260630
AISBench执行命令
ais_bench --models vllm_api_stream_chat --datasets gsm8k_gen_0_shot_cot_str_perf --debug --mode perf
模型配置文件或自定义配置文件内容
ais_bench --models vllm_api_stream_chat --datasets gsm8k_gen_0_shot_cot_str_perf --debug --mode perf
/benchmark/ais_bench/benchmark/configs/models/vllm_api/vllm_api_stream_chat.py：
from ais_bench.benchmark.models import VLLMCustomAPIChat
from ais_bench.benchmark.utils.postprocess.model_postprocessors import extract_non_reasoning_content
models = [
dict(
attr="service",
type=VLLMCustomAPIChat,
abbr='vllm-api-stream-chat',
path="/data/xxxx/qwen3.6-35b-a3b-w8a8",
model="qwen3.6-35b-a3b-dflash",
request_rate = 0,
retry = 2,
stream=True,
api_key="",
url="",
host_ip = "172.24.24.104",
host_port = 8750,
max_out_len = 1024,
batch_size= 1,
trust_remote_code=False,
generation_kwargs = dict(
ignore_eos = True,
temperature=0.01,
),
pred_postprocessor=dict(type=extract_non_reasoning_content)
)
]
预期行为
测试任务成功执行。
实际行为
[root@s104 Run_Benchmark]# ais_bench --models vllm_api_stream_chat --datasets gsm8k_gen_0_shot_cot_str_perf --debug --mode perf
[2026-07-17 06:40:38,022] [ais_bench] [INFO] Loading vllm_api_stream_chat: /benchmark/ais_bench/benchmark/configs/./models/vllm_api/vllm_api_stream_chat.py
[2026-07-17 06:40:38,027] [ais_bench] [INFO] Loading gsm8k_gen_0_shot_cot_str_perf: /benchmark/ais_bench/benchmark/configs/./datasets/gsm8k/gsm8k_gen_0_shot_cot_str_perf.py
[2026-07-17 06:40:38,030] [ais_bench] [INFO] Loading example: /benchmark/ais_bench/benchmark/configs/./summarizers/example.py
[2026-07-17 06:40:38,066] [ais_bench] [INFO] Current exp folder: outputs/default/20260717_064026
[2026-07-17 06:40:38,077] [ais_bench] [INFO] Starting inference tasks...
[2026-07-17 06:40:38,078] [ais_bench] [INFO] Partitioned into 1 tasks.
[2026-07-17 06:40:38,078] [ais_bench] [INFO] Merging datasets with the same model and inferencer...
[2026-07-17 06:40:38,116] [ais_bench] [INFO] Launch TasksMonitor, PID: 778271, Refresh interval: 0.5, Run in background: True
[2026-07-17 06:40:50,884] [ais_bench] [INFO] Debug mode, print progress directly
[2026-07-17 06:40:50,886] [ais_bench] [INFO] Task [vllm-api-stream-chat/gsm8k]
[2026-07-17 06:40:50,910] [ais_bench] [INFO] Zero Retriever initialized, returning empty shot case for all queries
[2026-07-17 06:40:50,910] [ais_bench] [INFO] Apply ice template finished
[2026-07-17 06:40:50,916] [ais_bench] [INFO] Start warmup, run with concurrency: 1
Warmup: 0%| | 0/1 [00:00<?, ?case/s]
卡死在warmup没有反应。
前置检查
操作系统及版本
oe2203.aarch64
安装工具的python环境
docker容器中的python环境
python版本
3.10
AISBench工具版本
v3.1-20260630
AISBench执行命令
ais_bench --models vllm_api_stream_chat --datasets gsm8k_gen_0_shot_cot_str_perf --debug --mode perf
模型配置文件或自定义配置文件内容
ais_bench --models vllm_api_stream_chat --datasets gsm8k_gen_0_shot_cot_str_perf --debug --mode perf
/benchmark/ais_bench/benchmark/configs/models/vllm_api/vllm_api_stream_chat.py：
from ais_bench.benchmark.models import VLLMCustomAPIChat
from ais_bench.benchmark.utils.postprocess.model_postprocessors import extract_non_reasoning_content
models = [
dict(
attr="service",
type=VLLMCustomAPIChat,
abbr='vllm-api-stream-chat',
path="/data/xxxx/qwen3.6-35b-a3b-w8a8",
model="qwen3.6-35b-a3b-dflash",
request_rate = 0,
retry = 2,
stream=True,
api_key="",
url="",
host_ip = "172.24.24.104",
host_port = 8750,
max_out_len = 1024,
batch_size= 1,
trust_remote_code=False,
generation_kwargs = dict(
ignore_eos = True,
temperature=0.01,
),
pred_postprocessor=dict(type=extract_non_reasoning_content)
)
]
预期行为
测试任务成功执行。
实际行为
[root@s104 Run_Benchmark]# ais_bench --models vllm_api_stream_chat --datasets gsm8k_gen_0_shot_cot_str_perf --debug --mode perf
[2026-07-17 06:40:38,022] [ais_bench] [INFO] Loading vllm_api_stream_chat: /benchmark/ais_bench/benchmark/configs/./models/vllm_api/vllm_api_stream_chat.py
[2026-07-17 06:40:38,027] [ais_bench] [INFO] Loading gsm8k_gen_0_shot_cot_str_perf: /benchmark/ais_bench/benchmark/configs/./datasets/gsm8k/gsm8k_gen_0_shot_cot_str_perf.py
[2026-07-17 06:40:38,030] [ais_bench] [INFO] Loading example: /benchmark/ais_bench/benchmark/configs/./summarizers/example.py
[2026-07-17 06:40:38,066] [ais_bench] [INFO] Current exp folder: outputs/default/20260717_064026
[2026-07-17 06:40:38,077] [ais_bench] [INFO] Starting inference tasks...
[2026-07-17 06:40:38,078] [ais_bench] [INFO] Partitioned into 1 tasks.
[2026-07-17 06:40:38,078] [ais_bench] [INFO] Merging datasets with the same model and inferencer...
[2026-07-17 06:40:38,116] [ais_bench] [INFO] Launch TasksMonitor, PID: 778271, Refresh interval: 0.5, Run in background: True
[2026-07-17 06:40:50,884] [ais_bench] [INFO] Debug mode, print progress directly
[2026-07-17 06:40:50,886] [ais_bench] [INFO] Task [vllm-api-stream-chat/gsm8k]
[2026-07-17 06:40:50,910] [ais_bench] [INFO] Zero Retriever initialized, returning empty shot case for all queries
[2026-07-17 06:40:50,910] [ais_bench] [INFO] Apply ice template finished
[2026-07-17 06:40:50,916] [ais_bench] [INFO] Start warmup, run with concurrency: 1
Warmup: 0%| | 0/1 [00:00<?, ?case/s]
卡死在warmup没有反应。
前置检查