[Bug]

[jack-SHL]

操作系统及版本

openEuler 24.03

安装工具的python环境

docker容器中的python环境

python版本

3.11

AISBench工具版本

3.1.20260415

AISBench执行命令

ais_bench --models vllm_api_general_chat --datasets textvqa_gen --debug -m perf

模型配置文件或自定义配置文件内容

models = [
dict(
attr="service",
type=VLLMCustomAPIChat,
abbr="vllm-api-general-chat",
path="/home/s60030007/Qwen3-VL-30B-A3B-Thinking",
model="qwen3-vl-30b",
stream=False,
request_rate=0,
use_timestamp=False,
retry=2,
api_key="",
host_ip="localhost",
host_port=8000,
url="",
max_out_len=1024,
batch_size=10,
trust_remote_code=False,
generation_kwargs=dict(
),
pred_postprocessor=dict(type=extract_non_reasoning_content),
)
]

预期行为

No response

实际行为

vllm报错如下：
(APIServer pid=6787) INFO 04-17 10:18:30 [chat_utils.py:590] Detected the chat template content format to be 'string'. You can set --chat-template-content-format to override this.
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] Error in preprocessing prompt inputs
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] Traceback (most recent call last):
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] File "/vllm-workspace/vllm/vllm/entrypoints/openai/serving_chat.py", line 237, in create_chat_completion
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] conversation, engine_prompts = await self._preprocess_chat(
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] File "/vllm-workspace/vllm/vllm/entrypoints/openai/serving_engine.py", line 1166, in _preprocess_chat
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] mm_data = await mm_data_future
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] ^^^^^^^^^^^^^^^^^^^^
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] File "/vllm-workspace/vllm/vllm/entrypoints/chat_utils.py", line 809, in all_mm_data
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] items_by_modality: dict[str, list[object | None]] = {
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] ^
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] File "/vllm-workspace/vllm/vllm/entrypoints/chat_utils.py", line 810, in
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] modality: await asyncio.gather(*coros)
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] File "/vllm-workspace/vllm/vllm/multimodal/utils.py", line 283, in fetch_image_async
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] return await self.load_from_url_async(
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] File "/vllm-workspace/vllm/vllm/multimodal/utils.py", line 208, in load_from_url_async
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] return await future
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] ^^^^^^^^^^^^
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] File "/usr/local/python3.11.14/lib/python3.11/concurrent/futures/thread.py", line 58, in run
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] result = self.fn(*self.args, **self.kwargs)
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] File "/vllm-workspace/vllm/vllm/multimodal/utils.py", line 128, in _load_file_url
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] raise ValueError(
(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] ValueError: The file path data/textvqa/train_images/003a8ae2ef43b901.jpg must be a subpath of --allowed-local-media-path /home/s60030007/benchmark/ais_bench/datasets/textvqa/train_images.
(APIServer pid=6787) INFO: 127.0.0.1:48724 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request

aisbench报错如下：
ais_bench --models vllm_api_general_chat --datasets textvqa_gen --debug -m perf
[2026-04-17 10:18:09,623] [ais_bench] [INFO] Loading vllm_api_general_chat: /home/s60030007/benchmark/ais_bench/benchmark/configs/./models/vllm_api/vllm_api_general_chat.py
[2026-04-17 10:18:09,629] [ais_bench] [INFO] Loading textvqa_gen: /home/s60030007/benchmark/ais_bench/benchmark/configs/./datasets/textvqa/textvqa_gen.py
[2026-04-17 10:18:09,632] [ais_bench] [INFO] Loading example: /home/s60030007/benchmark/ais_bench/benchmark/configs/./summarizers/example.py
[2026-04-17 10:18:09,681] [ais_bench] [INFO] Current exp folder: outputs/default/20260417_101757
[2026-04-17 10:18:09,696] [ais_bench] [INFO] Starting inference tasks...
[2026-04-17 10:18:09,697] [ais_bench] [INFO] Partitioned into 1 tasks.
[2026-04-17 10:18:09,697] [ais_bench] [INFO] Merging datasets with the same model and inferencer...
[2026-04-17 10:18:09,717] [ais_bench] [INFO] Launch TasksMonitor, PID: 8276, Refresh interval: 0.5, Run in background: True
[2026-04-17 10:18:24,271] [ais_bench] [INFO] Debug mode, print progress directly
[2026-04-17 10:18:24,273] [ais_bench] [INFO] Task [vllm-api-general-chat/textvqa]
[2026-04-17 10:18:25,589] [ais_bench] [INFO] Zero Retriever initialized, returning empty shot case for all queries
[2026-04-17 10:18:28,595] [ais_bench] [INFO] Apply ice template finished
[2026-04-17 10:18:29,215] [ais_bench] [INFO] Start warmup, run with concurrency: 10
Warmup: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1.16case/s]
[2026-04-17 10:18:30,081] [ais_bench] [INFO] Warmup finished Total Count: 1 Success Count: 0 Failed Count: 1
Failed Reasons:
+---------------+-------+
| Failed Reason | Count |
+---------------+-------+
| Bad Request | 1 |
+---------------+-------+
Traceback (most recent call last):
File "/home/s60030007/benchmark/ais_bench/benchmark/tasks/openicl_api_infer.py", line 704, in
raise e
File "/home/s60030007/benchmark/ais_bench/benchmark/tasks/openicl_api_infer.py", line 701, in
inferencer.run(task_state_manager)
File "/home/s60030007/benchmark/ais_bench/benchmark/tasks/openicl_api_infer.py", line 488, in run
self.warm_up(data_list, task_state_manager)
File "/home/s60030007/benchmark/ais_bench/benchmark/tasks/openicl_api_infer.py", line 467, in warm_up
raise AISBenchRuntimeError(
ais_bench.benchmark.utils.logging.exceptions.AISBenchRuntimeError: [TINFER-RUNTIME-001]warmup failed. Exit task because all warmup requests failed, failed reasons: {'Bad Request': 1} | Visit https://ais-bench-benchmark.readthedocs.io/zh-cn/latest/faqs/error_codes.html#tinfer-runtime-001 for further help.
[ERROR] 2026-04-17-10:18:30 (PID:8278, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception
[2026-04-17 10:18:32,711] [ais_bench] [INFO] Inference tasks completed.
[2026-04-17 10:18:32,713] [ais_bench] [INFO] Summarizing performance results...
[2026-04-17 10:18:32,713] [ais_bench] [WARNING] Can't find details perf data of [vllm-api-general-chat/textvqa] in outputs/default/20260417_101757, use tmp cache data.
Traceback (most recent call last):
File "/usr/local/python3.11.14/bin/ais_bench", line 7, in
sys.exit(main())
^^^^^^
File "/home/s60030007/benchmark/ais_bench/benchmark/cli/main.py", line 6, in main
task_manager.run()
File "/home/s60030007/benchmark/ais_bench/benchmark/cli/task_manager.py", line 39, in run
workflow_executor.execute()
File "/home/s60030007/benchmark/ais_bench/benchmark/cli/workers.py", line 492, in execute
worker.do_work(cfg)
File "/home/s60030007/benchmark/ais_bench/benchmark/cli/workers.py", line 469, in do_work
summarizer.summarize()
File "/home/s60030007/benchmark/ais_bench/benchmark/summarizers/default_perf.py", line 427, in summarize
details_perf_datas = self._load_details_perf_data(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/s60030007/benchmark/ais_bench/benchmark/summarizers/default_perf.py", line 251, in _load_details_perf_data
raise FileMatchError(
ais_bench.benchmark.utils.logging.exceptions.FileMatchError: [SUMM-FILE-001]can't find detail perf data file. Can't find any details perf data file in work_dir, please check outputs/default/20260417_101757. | Visit https://ais-bench-benchmark.readthedocs.io/zh-cn/latest/faqs/error_codes.html#summ-file-001 for further help.
[ERROR] 2026-04-17-10:18:32 (PID:8202, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception

前置检查

• 我已读懂主页文档的快速入门，无法解决问题
• 我已检索过FAQ，无重复问题
• 我已搜索过现有Issue，无重复问题
• 我已更新到最新版本，问题仍存在

[github-actions]

🌟 Welcome to file an issue for the AISBench benchmark tool. You can try the following ways to find solutions to your issue:

1. [Strongly recommended❤️‍🔥]Ensure your issue description is complete, then try to ask for help, as deepwiki contains all the knowledge related to the tool
2. Search for similar issues in the tool's historical issues using key log fragments or keywords
3. Search in the official documentation search bar using key log fragments or keywords
4. Seek help from repository maintainers (mention @SJTUyh in the issue comment)

[github-actions]

🤖 基于AI机器人的issue内容完整性检查结果:

问题内容检查结果

检查状态

PASS

详细检查

• 操作系统及版本: COMPLETE
• 安装工具的python环境: COMPLETE
• python版本: COMPLETE
• AISBench工具版本: COMPLETE
• AISBench执行命令: COMPLETE
• 模型配置文件或自定义配置文件内容: COMPLETE
• 实际行为: COMPLETE

改进建议

所有内容已完备！

补充说明

该Issue内容完整，所有要求的部分都已提供，并且信息详细、相关。

👉 如果想重新检查，在评论区@issue_checker即可。

【强烈推荐❤️‍🔥】确保issue描述完整后，可以试着将issue交给回答，deepwiki包含了和工具相关的所有知识库

[SJTUyh]

(APIServer pid=6787) ERROR 04-17 10:18:30 [serving_chat.py:255] ValueError: The file path data/textvqa/train_images/003a8ae2ef43b901.jpg must be a subpath of --allowed-local-media-path /home/s60030007/benchmark/ais_bench/datasets/textvqa/train_images.

你的服务日志中限定了/home/s60030007/benchmark/ais_bench/datasets/textvqa/train_images这个绝对路径，图片路径必须显式得是其子路径，你的图片路径必须是绝对路径。
修改https://github.com/AISBench/benchmark/blob/master/ais_bench/benchmark/configs/datasets/textvqa/textvqa_gen.py这个文件中

textvqa_infer_cfg = dict(
    prompt_template=dict(
        type=MMPromptTemplate,
        template=dict(
            round=[
                dict(role="HUMAN", prompt_mm={
                    "text": {"type": "text", "text": "{question} Answer the question using a single word or phrase."},
                    # "image": {"type": "image_url", "image_url": {"url": "file://{image}"}},
                    "image": {"type": "image_url", "image_url": {"url": "file://`/home/s60030007/benchmark/ais_bench/datasets/textvqa/train_images/{image}"}}, # 加上绝对路径
                })
            ]
            )
    ),
    retriever=dict(type=ZeroRetriever),
    inferencer=dict(type=GenInferencer)
)