Skip to content

Add Quantization Support for Qwen2.5-VL Model #270

@kurosakiharachan

Description

@kurosakiharachan

The https://github.com/NVIDIA/TensorRT-Model-Optimizer/tree/main/examples/vlm_ptq page indicates that TensorRT-Model-Optimizer currently only supports quantization for three multimodal models: Llava, VILA, and Phi-3-vision. It would be beneficial to add support for quantizing the Qwen2.5-VL model as well.
Image

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions