latest/version3.x/pipeline_usage/PaddleOCR-VL-NVIDIA-Blackwell

[giscus[bot]]

latest/version3.x/pipeline_usage/PaddleOCR-VL-NVIDIA-Blackwell

Awesome multilingual OCR toolkits based on PaddlePaddle (practical ultra lightweight OCR system, support 80+ languages recognition, provide data annotation and synthesis tools, support training and deployment among server, mobile, embedded and IoT devices)

https://www.paddleocr.ai/latest/version3.x/pipeline_usage/PaddleOCR-VL-NVIDIA-Blackwell.html

[GitCiris]

(base) hadoop@LAPTOP-IUTU7052:~$ paddleocr genai_server --model_name PaddleOCR-VL-1.5-0.9B --backend vllm --port 8118
Checking connectivity to the model hosters, this may take a while. To bypass this check, set PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK to True.
Using official model (PaddleOCR-VL-1.5), the model files will be automatically downloaded and saved in /home/hadoop/.paddlex/official_models/PaddleOCR-VL-1.5.
Downloading [README.md]: 100%|█████████████████████████████████████████████████████| 22.3k/22.3k [00:00<00:00, 84.8kB/s]
Downloading [LICENSE]: 100%|███████████████████████████████████████████████████████| 11.1k/11.1k [00:00<00:00, 23.5kB/s]
Downloading [added_tokens.json]: 100%|█████████████████████████████████████████████| 24.8k/24.8k [00:00<00:00, 25.9kB/s]
Downloading [chat_template.jinja]: 100%|███████████████████████████████████████████| 1.44k/1.44k [00:00<00:00, 3.01kB/s]
Downloading [config.json]: 100%|███████████████████████████████████████████████████| 2.01k/2.01k [00:00<00:00, 2.61kB/s]
Downloading [configuration_paddleocr_vl.py]: 100%|█████████████████████████████████| 7.91k/7.91k [00:00<00:00, 15.5kB/s]
Downloading [generation_config.json]: 100%|██████████████████████████████████████████████| 133/133 [00:01<00:00, 133B/s]
Downloading [inference.yml]: 100%|████████████████████████████████████████████████████| 43.0/43.0 [00:00<00:00, 84.1B/s]
Downloading [image_processing_paddleocr_vl.py]: 100%|██████████████████████████████| 24.4k/24.4k [00:01<00:00, 24.5kB/s]
Downloading [modeling_paddleocr_vl.py]: 100%|█████████████████████████████████████████| 109k/109k [00:00<00:00, 223kB/s]
Downloading [processing_paddleocr_vl.py]: 100%|████████████████████████████████████| 12.0k/12.0k [00:00<00:00, 15.6kB/s]
Downloading [preprocessor_config.json]: 100%|████████████████████████████████████████████| 641/641 [00:00<00:00, 743B/s]
Downloading [processor_config.json]: 100%|███████████████████████████████████████████████| 137/137 [00:01<00:00, 117B/s]
Downloading [special_tokens_map.json]: 100%|███████████████████████████████████████| 1.12k/1.12k [00:01<00:00, 1.11kB/s]
Downloading [tokenizer.json]: 100%|████████████████████████████████████████████████| 10.7M/10.7M [00:02<00:00, 4.29MB/s]
Downloading [tokenizer.model]: 100%|████████████████████████████████████████████████| 1.54M/1.54M [00:02<00:00, 676kB/s]
Downloading [tokenizer_config.json]: 100%|████████████████████████████████████████████| 183k/183k [00:01<00:00, 102kB/s]
Downloading [model.safetensors]: 100%|█████████████████████████████████████████████| 1.79G/1.79G [00:52<00:00, 36.8MB/s]
Processing 18 items: 100%|███████████████████████████████████████████████████████████| 18.0/18.0 [00:59<00:00, 3.28s/it]
INFO 02-07 13:58:57 [init.py:216] Automatically detected platform cuda.
WARNING 02-07 13:59:00 [registry.py:483] Model architecture PaddleOCRVLForConditionalGeneration is already registered, and will be overwritten by the new model class <class 'paddlex.inference.genai.models.paddleocr_vl_09b._vllm.PaddleOCRVLForConditionalGeneration'>.
(APIServer pid=9897) INFO 02-07 13:59:00 [api_server.py:1896] vLLM API server version 0.10.2
(APIServer pid=9897) INFO 02-07 13:59:00 [utils.py:328] non-default args: {'api_server_count': 4, 'host': 'localhost', 'port': 8118, 'chat_template': '/home/hadoop/miniconda3/lib/python3.12/site-packages/paddlex/inference/genai/chat_templates/PaddleOCR-VL-1.5-0.9B.jinja', 'model': '/home/hadoop/.paddlex/official_models/PaddleOCR-VL-1.5', 'trust_remote_code': True, 'max_model_len': 16384, 'served_model_name': ['PaddleOCR-VL-1.5-0.9B'], 'gpu_memory_utilization': 0.5, 'max_num_batched_tokens': 131072}
(APIServer pid=9897) The argument trust_remote_code is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=9897) INFO 02-07 13:59:00 [init.py:742] Resolved architecture: PaddleOCRVLForConditionalGeneration
(APIServer pid=9897) torch_dtype is deprecated! Use dtype instead!
(APIServer pid=9897) INFO 02-07 13:59:00 [init.py:1815] Using max model len 16384
(APIServer pid=9897) INFO 02-07 13:59:00 [scheduler.py:222] Chunked prefill is enabled with max_num_batched_tokens=131072.
Checking connectivity to the model hosters, this may take a while. To bypass this check, set PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK to True.
INFO 02-07 13:59:07 [init.py:216] Automatically detected platform cuda.
(EngineCore_DP0 pid=10014) INFO 02-07 13:59:08 [core.py:654] Waiting for init message from front-end.
(EngineCore_DP0 pid=10014) WARNING 02-07 13:59:09 [registry.py:483] Model architecture PaddleOCRVLForConditionalGeneration is already registered, and will be overwritten by the new model class <class 'paddlex.inference.genai.models.paddleocr_vl_09b._vllm.PaddleOCRVLForConditionalGeneration'>.
(EngineCore_DP0 pid=10014) INFO 02-07 13:59:09 [core.py:76] Initializing a V1 LLM engine (v0.10.2) with config: model='/home/hadoop/.paddlex/official_models/PaddleOCR-VL-1.5', speculative_config=None, tokenizer='/home/hadoop/.paddlex/official_models/PaddleOCR-VL-1.5', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=16384, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, decoding_config=DecodingConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=PaddleOCR-VL-1.5-0.9B, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=True, pooler_config=None, compilation_config={"level":3,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":[],"splitting_ops":["vllm.unified_attention","vllm.unified_attention_with_output","vllm.mamba_mixer2","vllm.mamba_mixer","vllm.short_conv","vllm.linear_attention","vllm.plamo2_mamba_mixer","vllm.gdn_attention"],"use_inductor":true,"compile_sizes":[],"inductor_compile_config":{"enable_auto_functionalized_v2":false},"inductor_passes":{},"cudagraph_mode":1,"use_cudagraph":true,"cudagraph_num_of_warmups":1,"cudagraph_capture_sizes":[512,504,496,488,480,472,464,456,448,440,432,424,416,408,400,392,384,376,368,360,352,344,336,328,320,312,304,296,288,280,272,264,256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"pass_config":{},"max_capture_size":512,"local_cache_dir":null}
(EngineCore_DP0 pid=10014) WARNING 02-07 13:59:09 [interface.py:391] Using 'pin_memory=False' as WSL is detected. This may slow down the performance.
[W207 13:59:09.689378517 ProcessGroupNCCL.cpp:981] Warning: TORCH_NCCL_AVOID_RECORD_STREAMS is the default now, this environment variable is thus deprecated. (function operator())
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
(EngineCore_DP0 pid=10014) INFO 02-07 13:59:09 [parallel_state.py:1165] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=10014) WARNING 02-07 13:59:10 [topk_topp_sampler.py:69] FlashInfer is not available. Falling back to the PyTorch-native implementation of top-p & top-k sampling. For the best performance, please install FlashInfer.
(EngineCore_DP0 pid=10014) Using a slow image processor as use_fast is unset and a slow processor was saved with this model. use_fast=True will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with use_fast=False.
(EngineCore_DP0 pid=10014) INFO 02-07 13:59:12 [gpu_model_runner.py:2338] Starting to load model /home/hadoop/.paddlex/official_models/PaddleOCR-VL-1.5...
(EngineCore_DP0 pid=10014) INFO 02-07 13:59:12 [gpu_model_runner.py:2370] Loading model from scratch...
(EngineCore_DP0 pid=10014) INFO 02-07 13:59:12 [cuda.py:362] Using Flash Attention backend on V1 engine.
Loading safetensors checkpoint shards: 0% Completed | 0/1 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00, 2.96it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00, 2.96it/s]
(EngineCore_DP0 pid=10014)
(EngineCore_DP0 pid=10014) INFO 02-07 13:59:13 [default_loader.py:268] Loading weights took 0.42 seconds
(EngineCore_DP0 pid=10014) INFO 02-07 13:59:13 [gpu_model_runner.py:2392] Model loading took 1.8939 GiB and 0.641668 seconds
(EngineCore_DP0 pid=10014) INFO 02-07 13:59:14 [gpu_model_runner.py:3000] Encoder cache will be initialized with a budget of 131072 tokens, and profiled with 668 image items of the maximum feature size.
(EngineCore_DP0 pid=10014) INFO 02-07 13:59:40 [backends.py:539] Using cache directory: /home/hadoop/.cache/vllm/torch_compile_cache/ab8eb0024c/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=10014) INFO 02-07 13:59:40 [backends.py:550] Dynamo bytecode transform time: 2.50 s
(EngineCore_DP0 pid=10014) [rank0]:W0207 13:59:40.095000 10014 site-packages/torch/_inductor/utils.py:1436] [0/0] Not enough SMs to use max_autotune_gemm mode
(EngineCore_DP0 pid=10014) INFO 02-07 13:59:41 [backends.py:194] Cache the graph for dynamic shape for later use
(EngineCore_DP0 pid=10014) INFO 02-07 13:59:50 [backends.py:215] Compiling a graph for dynamic shape takes 10.19 s
(EngineCore_DP0 pid=10014) INFO 02-07 13:59:52 [monitor.py:34] torch.compile takes 12.69 s in total
(EngineCore_DP0 pid=10014) INFO 02-07 13:59:59 [gpu_worker.py:298] Available KV cache memory: -2.26 GiB
(EngineCore_DP0 pid=10014) ERROR 02-07 13:59:59 [core.py:718] EngineCore failed to start.
(EngineCore_DP0 pid=10014) ERROR 02-07 13:59:59 [core.py:718] Traceback (most recent call last):
(EngineCore_DP0 pid=10014) ERROR 02-07 13:59:59 [core.py:718] File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 709, in run_engine_core
(EngineCore_DP0 pid=10014) ERROR 02-07 13:59:59 [core.py:718] engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=10014) ERROR 02-07 13:59:59 [core.py:718] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=10014) ERROR 02-07 13:59:59 [core.py:718] File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 505, in init
(EngineCore_DP0 pid=10014) ERROR 02-07 13:59:59 [core.py:718] super().init(vllm_config, executor_class, log_stats,
(EngineCore_DP0 pid=10014) ERROR 02-07 13:59:59 [core.py:718] File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 91, in init
(EngineCore_DP0 pid=10014) ERROR 02-07 13:59:59 [core.py:718] self._initialize_kv_caches(vllm_config)
(EngineCore_DP0 pid=10014) ERROR 02-07 13:59:59 [core.py:718] File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 193, in _initialize_kv_caches
(EngineCore_DP0 pid=10014) ERROR 02-07 13:59:59 [core.py:718] get_kv_cache_config(vllm_config, kv_cache_spec_one_worker,
(EngineCore_DP0 pid=10014) ERROR 02-07 13:59:59 [core.py:718] File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/core/kv_cache_utils.py", line 1110, in get_kv_cache_config
(EngineCore_DP0 pid=10014) ERROR 02-07 13:59:59 [core.py:718] check_enough_kv_cache_memory(vllm_config, kv_cache_spec, available_memory)
(EngineCore_DP0 pid=10014) ERROR 02-07 13:59:59 [core.py:718] File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/core/kv_cache_utils.py", line 691, in check_enough_kv_cache_memory
(EngineCore_DP0 pid=10014) ERROR 02-07 13:59:59 [core.py:718] raise ValueError("No available memory for the cache blocks. "
(EngineCore_DP0 pid=10014) ERROR 02-07 13:59:59 [core.py:718] ValueError: No available memory for the cache blocks. Try increasing gpu_memory_utilization when initializing the engine.
(EngineCore_DP0 pid=10014) Process EngineCore_DP0:
(EngineCore_DP0 pid=10014) Traceback (most recent call last):
(EngineCore_DP0 pid=10014) File "/home/hadoop/miniconda3/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore_DP0 pid=10014) self.run()
(EngineCore_DP0 pid=10014) File "/home/hadoop/miniconda3/lib/python3.12/multiprocessing/process.py", line 108, in run
(EngineCore_DP0 pid=10014) self._target(*self._args, **self._kwargs)
(EngineCore_DP0 pid=10014) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 722, in run_engine_core
(EngineCore_DP0 pid=10014) raise e
(EngineCore_DP0 pid=10014) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 709, in run_engine_core
(EngineCore_DP0 pid=10014) engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=10014) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=10014) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 505, in init
(EngineCore_DP0 pid=10014) super().init(vllm_config, executor_class, log_stats,
(EngineCore_DP0 pid=10014) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 91, in init
(EngineCore_DP0 pid=10014) self._initialize_kv_caches(vllm_config)
(EngineCore_DP0 pid=10014) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 193, in _initialize_kv_caches
(EngineCore_DP0 pid=10014) get_kv_cache_config(vllm_config, kv_cache_spec_one_worker,
(EngineCore_DP0 pid=10014) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/core/kv_cache_utils.py", line 1110, in get_kv_cache_config
(EngineCore_DP0 pid=10014) check_enough_kv_cache_memory(vllm_config, kv_cache_spec, available_memory)
(EngineCore_DP0 pid=10014) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/core/kv_cache_utils.py", line 691, in check_enough_kv_cache_memory
(EngineCore_DP0 pid=10014) raise ValueError("No available memory for the cache blocks. "
(EngineCore_DP0 pid=10014) ValueError: No available memory for the cache blocks. Try increasing gpu_memory_utilization when initializing the engine.
[rank0]:[W207 14:00:00.530535562 ProcessGroupNCCL.cpp:1538] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
(APIServer pid=9897) Traceback (most recent call last):
(APIServer pid=9897) File "/home/hadoop/miniconda3/bin/paddleocr", line 7, in
(APIServer pid=9897) sys.exit(console_entry())
(APIServer pid=9897) ^^^^^^^^^^^^^^^
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/paddleocr/main.py", line 26, in console_entry
(APIServer pid=9897) main()
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/paddleocr/_cli.py", line 194, in main
(APIServer pid=9897) _execute(args)
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/paddleocr/_cli.py", line 183, in _execute
(APIServer pid=9897) args.executor(args)
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/paddleocr/_cli.py", line 157, in _run_genai_server
(APIServer pid=9897) run_genai_server(args)
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/paddlex/inference/genai/server.py", line 100, in run_genai_server
(APIServer pid=9897) run_server_func(
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/paddlex/inference/genai/backends/vllm.py", line 88, in run_vllm_server
(APIServer pid=9897) uvloop.run(run_server(args))
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/uvloop/init.py", line 96, in run
(APIServer pid=9897) return __asyncio.run(
(APIServer pid=9897) ^^^^^^^^^^^^^^
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/asyncio/runners.py", line 195, in run
(APIServer pid=9897) return runner.run(main)
(APIServer pid=9897) ^^^^^^^^^^^^^^^^
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/asyncio/runners.py", line 118, in run
(APIServer pid=9897) return self._loop.run_until_complete(task)
(APIServer pid=9897) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=9897) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/uvloop/init.py", line 48, in wrapper
(APIServer pid=9897) return await main
(APIServer pid=9897) ^^^^^^^^^^
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 1941, in run_server
(APIServer pid=9897) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 1961, in run_server_worker
(APIServer pid=9897) async with build_async_engine_client(
(APIServer pid=9897) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/contextlib.py", line 210, in aenter
(APIServer pid=9897) return await anext(self.gen)
(APIServer pid=9897) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 179, in build_async_engine_client
(APIServer pid=9897) async with build_async_engine_client_from_engine_args(
(APIServer pid=9897) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/contextlib.py", line 210, in aenter
(APIServer pid=9897) return await anext(self.gen)
(APIServer pid=9897) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 221, in build_async_engine_client_from_engine_args
(APIServer pid=9897) async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=9897) ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/utils/init.py", line 1589, in inner
(APIServer pid=9897) return fn(*args, **kwargs)
(APIServer pid=9897) ^^^^^^^^^^^^^^^^^^^
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py", line 212, in from_vllm_config
(APIServer pid=9897) return cls(
(APIServer pid=9897) ^^^^
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/engine/async_llm.py", line 136, in init
(APIServer pid=9897) self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=9897) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 102, in make_async_mp_client
(APIServer pid=9897) return AsyncMPClient(*client_args)
(APIServer pid=9897) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 769, in init
(APIServer pid=9897) super().init(
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 448, in init
(APIServer pid=9897) with launch_core_engines(vllm_config, executor_class,
(APIServer pid=9897) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/contextlib.py", line 144, in exit
(APIServer pid=9897) next(self.gen)
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/engine/utils.py", line 729, in launch_core_engines
(APIServer pid=9897) wait_for_engine_startup(
(APIServer pid=9897) File "/home/hadoop/miniconda3/lib/python3.12/site-packages/vllm/v1/engine/utils.py", line 782, in wait_for_engine_startup
(APIServer pid=9897) raise RuntimeError("Engine core initialization failed. "
(APIServer pid=9897) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}

[wzz981]

想问一下这些镜像怎么使用root权限，想移动文件都会报Permission denied