[Bug]: Llama4 multimodal cache missing key #20203
Description
Your current environment
Python 3.11 CUDA 12.4 vLLM 0.8.4
🐛 Describe the bug
vLLM v0.8.4
ERROR 06-27 16:32:19 [core.py:387] EngineCore hit an exception: Traceback (most recent call last):
ERROR 06-27 16:32:19 [core.py:387] File "/opt/conda/lib/python3.11/site-packages/cachetools/__init__.py", line 66, in __getitem__
ERROR 06-27 16:32:19 [core.py:387] return self.__data[key]
ERROR 06-27 16:32:19 [core.py:387] ~~~~~~~~~~~^^^^^
ERROR 06-27 16:32:19 [core.py:387] KeyError: 'f224b2249792ee5054e8ec7c0b68f6d22889843159c73bdea187eda555743f85'
ERROR 06-27 16:32:19 [core.py:387]
ERROR 06-27 16:32:19 [core.py:387] During handling of the above exception, another exception occurred:
ERROR 06-27 16:32:19 [core.py:387]
ERROR 06-27 16:32:19 [core.py:387] Traceback (most recent call last):
ERROR 06-27 16:32:19 [core.py:387] File "/opt/conda/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 380, in run_engine_core
ERROR 06-27 16:32:19 [core.py:387] engine_core.run_busy_loop()
ERROR 06-27 16:32:19 [core.py:387] File "/opt/conda/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 400, in run_busy_loop
ERROR 06-27 16:32:19 [core.py:387] self._process_input_queue()
ERROR 06-27 16:32:19 [core.py:387] File "/opt/conda/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 425, in _process_input_queue
ERROR 06-27 16:32:19 [core.py:387] self._handle_client_request(*req)
ERROR 06-27 16:32:19 [core.py:387] File "/opt/conda/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 441, in _handle_client_request
ERROR 06-27 16:32:19 [core.py:387] self.add_request(request)
ERROR 06-27 16:32:19 [core.py:387] File "/opt/conda/lib/python3.11/site-packages/vllm/v1/engine/core.py", line 177, in add_request
ERROR 06-27 16:32:19 [core.py:387] request.mm_inputs = self.mm_input_cache_server.get_and_update_p1(
ERROR 06-27 16:32:19 [core.py:387] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-27 16:32:19 [core.py:387] File "/opt/conda/lib/python3.11/site-packages/vllm/v1/engine/mm_input_cache.py", line 76, in get_and_update_p1
ERROR 06-27 16:32:19 [core.py:387] mm_input = self.mm_cache[mm_hash]
ERROR 06-27 16:32:19 [core.py:387] ~~~~~~~~~~~~~^^^^^^^^^
ERROR 06-27 16:32:19 [core.py:387] File "/opt/conda/lib/python3.11/site-packages/cachetools/__init__.py", line 255, in __getitem__
ERROR 06-27 16:32:19 [core.py:387] value = cache_getitem(self, key)
ERROR 06-27 16:32:19 [core.py:387] ^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-27 16:32:19 [core.py:387] File "/opt/conda/lib/python3.11/site-packages/cachetools/__init__.py", line 68, in __getitem__
ERROR 06-27 16:32:19 [core.py:387] return self.__missing__(key)
ERROR 06-27 16:32:19 [core.py:387] ^^^^^^^^^^^^^^^^^^^^^
ERROR 06-27 16:32:19 [core.py:387] File "/opt/conda/lib/python3.11/site-packages/cachetools/__init__.py", line 95, in __missing__
ERROR 06-27 16:32:19 [core.py:387] raise KeyError(key)
ERROR 06-27 16:32:19 [core.py:387] KeyError: 'f224b2249792ee5054e8ec7c0b68f6d22889843159c73bdea187eda555743f85'
ERROR 06-27 16:32:19 [core.py:387]
CRITICAL 06-27 16:32:19 [core_client.py:359] Got fatal signal from worker processes, shutting down. See stack trace above for root cause issue.
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.