How to input multimodal embeddings to an LLM

[RuABraun]

I'm using this function to try and pass multimodal embeddings: https://github.com/NVIDIA/TensorRT-LLM/blob/release/0.20/tensorrt_llm/llmapi/llm.py#L341

I'm assuming the fusing behaviour to be like here: https://github.com/NVIDIA/TensorRT-LLM/blob/release/0.20/tensorrt_llm/_torch/models/modeling_multimodal_utils.py#L31

Is my thinking accurate? Based on what nonsense my model outputs I assume no. I have not been able to find relevant docs or figure out where the C++ is