Skip to content

Commit 744f0ef

Browse files
authored
[TRTLLM-9522][fix] restore trtllm-serve mm_embedding_serve (#9669)
1 parent 9492463 commit 744f0ef

File tree

2 files changed

+4
-1
lines changed

2 files changed

+4
-1
lines changed

tensorrt_llm/commands/serve.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -216,12 +216,14 @@ def launch_mm_encoder_server(
216216
metadata_server_cfg: Optional[MetadataServerConfig] = None,
217217
):
218218
model = encoder_args["model"]
219+
encoder_args.pop("build_config")
219220
mm_encoder = MultimodalEncoder(**encoder_args)
220221

221222
server = OpenAIServer(llm=mm_encoder,
222223
model=model,
223224
server_role=ServerRole.MM_ENCODER,
224-
metadata_server_cfg=metadata_server_cfg)
225+
metadata_server_cfg=metadata_server_cfg,
226+
tool_parser=None)
225227
asyncio.run(server(host, port))
226228

227229

tests/integration/test_lists/test-db/l0_a10.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@ l0_a10:
5050
- disaggregated/test_disaggregated_single_gpu.py::test_disaggregated_simple_llama[True-True-TinyLlama-1.1B-Chat-v1.0]
5151
- test_e2e.py::test_openai_chat_guided_decoding
5252
- test_e2e.py::test_openai_chat_multimodal_example ISOLATION
53+
- test_e2e.py::test_openai_mmencoder_example
5354
- test_e2e.py::test_openai_perf_metrics
5455
- test_e2e.py::test_openai_prometheus
5556
- test_e2e.py::test_openai_lora

0 commit comments

Comments
 (0)