Your current environment

ValueError: The input size is not aligned with the quantized weight shape. This can be caused by too large tensor parallel size.

Model name - Ovis2.5 9B
Method - gptq

🐛 Describe the bug

ValueError: The input size is not aligned with the quantized weight shape. This can be caused by too large tensor parallel size.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.