vllm 启动模型服务报错

[wwenqi-cn]

使用Readme中的代码进行图片的识别

from PIL import Image
from transformers import AutoProcessor
from vllm import LLM, SamplingParams

model_path = "moonshotai/Kimi-VL-A3B-Instruct"  # or "moonshotai/Kimi-VL-A3B-Thinking-2506"
llm = LLM(
    model_path,
    trust_remote_code=True,
)

processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)

image_path = "./figures/demo.png"
image = Image.open(image_path)
messages = [
    {"role": "user", "content": [{"type": "image", "image": image_path}, {"type": "text", "text": "What is the dome building in the picture? Think step by step."}]}
]
text = processor.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = llm.generate([{"prompt": text, "multi_modal_data": {"image": image}}], sampling_params = SamplingParams(max_tokens=512))

print("-" * 50)
for o in outputs:
    generated_text = o.outputs[0].text
    print(generated_text)
    print("-" * 50)

报错

Traceback (most recent call last):
  File "/moonshotai/test.py", line 3, in <module>
    from vllm import LLM, SamplingParams
  File "/opt/conda/lib/python3.11/site-packages/vllm/__init__.py", line 13, in <module>
    from vllm.engine.arg_utils import AsyncEngineArgs, EngineArgs
  File "/opt/conda/lib/python3.11/site-packages/vllm/engine/arg_utils.py", line 22, in <module>
    from vllm.config import (BlockSize, CacheConfig, CacheDType, CompilationConfig,
  File "/opt/conda/lib/python3.11/site-packages/vllm/config.py", line 43, in <module>
    from vllm.transformers_utils.config import (
  File "/opt/conda/lib/python3.11/site-packages/vllm/transformers_utils/config.py", line 33, in <module>
    from vllm.transformers_utils.configs import (ChatGLMConfig, Cohere2Config,
  File "/opt/conda/lib/python3.11/site-packages/vllm/transformers_utils/configs/__init__.py", line 28, in <module>
    from vllm.transformers_utils.configs.ovis import OvisConfig
  File "/opt/conda/lib/python3.11/site-packages/vllm/transformers_utils/configs/ovis.py", line 76, in <module>
    AutoConfig.register("aimv2", AIMv2Config)
  File "/opt/conda/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1306, in register
    CONFIG_MAPPING.register(model_type, config, exist_ok=exist_ok)
  File "/opt/conda/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 993, in register
    raise ValueError(f"'{key}' is already used by a Transformers config, pick another name.")
ValueError: 'aimv2' is already used by a Transformers config, pick another name.

同时尝试直接用vllm启动服务，调用第一次时不会报错，第二次会报错

VllmWorker rank=0 pid=7350) ERROR 08-01 01:08:29 [multiproc_executor.py:470]     return func(*args, **kwargs)
(VllmWorker rank=3 pid=7353) ERROR 08-01 01:08:29 [multiproc_executor.py:470]     return func(*args, **kwargs)
(VllmWorker rank=0 pid=7350) ERROR 08-01 01:08:29 [multiproc_executor.py:470]            ^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=3 pid=7353) ERROR 08-01 01:08:29 [multiproc_executor.py:470]            ^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=7350) ERROR 08-01 01:08:29 [multiproc_executor.py:470]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 268, in execute_model
(VllmWorker rank=3 pid=7353) ERROR 08-01 01:08:29 [multiproc_executor.py:470]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 268, in execute_model
(VllmWorker rank=0 pid=7350) ERROR 08-01 01:08:29 [multiproc_executor.py:470]     output = self.model_runner.execute_model(scheduler_output)
(VllmWorker rank=3 pid=7353) ERROR 08-01 01:08:29 [multiproc_executor.py:470]     output = self.model_runner.execute_model(scheduler_output)
(VllmWorker rank=0 pid=7350) ERROR 08-01 01:08:29 [multiproc_executor.py:470]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=3 pid=7353) ERROR 08-01 01:08:29 [multiproc_executor.py:470]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=7350) ERROR 08-01 01:08:29 [multiproc_executor.File "/usr/local/lib/python3.12/dist-packages/vllm/v1/attention/backends/mla/common.py", line 504, in build
(VllmWorker rank=3 pid=7353) ERROR 08-01 01:08:29 [multiproc_executor.py:470]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/attention/backends/mla/common.py", line 504, in build
(VllmWorker rank=0 pid=7350) ERROR 08-01 01:08:29 [multiproc_executor.py:470]     self.page_size)
(VllmWorker rank=3 pid=7353) ERROR 08-01 01:08:29 [multiproc_executor.py:470]     self.page_size)
(VllmWorker rank=0 pid=7350) ERROR 08-01 01:08:29 [multiproc_executor.py:470]     ^^^^^^^^^^^^^^
(VllmWorker rank=3 pid=7353) ERROR 08-01 01:08:29 [multiproc_executor.py:470]     ^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=7350) ERROR 08-01 01:08:29 [multiproc_executor.py:470] AttributeError: 'MLACommonMetadataBuilder' object has no attribute 'page_size'
(VllmWorker rank=3 pid=7353) ERROR 08-01 01:08:29 [multiproc_executor.py:470] AttributeError: 'MLACommonMetadataBuilder' object has no attribute 'page_size'

想请教一下是什么原因， vllm 0.8.5和vllm 0.9.1 都尝试了，模型是 Kimi-VL-A3B-Instruct

[shimiaoyuan]

@wwenqi-cn VLLM use 0.9.2，and this error caused by transformers version, use transformers==4.52.4

[wangwenqi567]

感谢回复