Skip to content

Commit cad6ed8

Browse files
authored
docs: add addt vision model support (#546)
* docs: add addt vision model support Signed-off-by: Anh Uong <[email protected]> * update llava-1.5-13b checkmark Signed-off-by: Anh Uong <[email protected]> * update release version and remove llava next model Signed-off-by: Anh Uong <[email protected]> --------- Signed-off-by: Anh Uong <[email protected]>
1 parent f2ba2c3 commit cad6ed8

File tree

1 file changed

+9
-6
lines changed

1 file changed

+9
-6
lines changed

README.md

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -909,6 +909,8 @@ For information on supported dataset formats and how to tune a vision-language m
909909

910910
### Supported vision model
911911

912+
Note that vision models are supported starting with `fms-hf-tuning` v2.8.1 or later.
913+
912914
- Legend:
913915

914916
✅ Ready and available
@@ -921,12 +923,13 @@ For information on supported dataset formats and how to tune a vision-language m
921923

922924
Model Name & Size | Model Architecture | LoRA Tuning | Full Finetuning |
923925
-------------------- | ---------------- | --------------- | --------------- |
924-
Llama 3.2-11B Vision | MllamaForConditionalGeneration | ✅* | ✅* |
925-
Llava 1.5-7B | LlavaForConditionalGeneration | ✅* | ✅* |
926-
Granite 3.1-2B Vision | LlavaNextForConditionalGeneration | ✅* | ✅* |
927-
Llava Mistral 1.6-7B | LlavaNextForConditionalGeneration | ✅* | ✅* |
928-
929-
(*) - Supported with `fms-hf-tuning` v2.8.0 or later.
926+
Llama 3.2-11B Vision | MllamaForConditionalGeneration | ✅ | ✅ |
927+
Llama 3.2-90B Vision | MllamaForConditionalGeneration | ✔️ | ✔️ |
928+
Granite 3.2-2B Vision | LlavaNextForConditionalGeneration | ✅ | ✅ |
929+
Llava Mistral 1.6-7B | LlavaNextForConditionalGeneration | ✅ | ✅ |
930+
Llava 1.6-34B | LlavaNextForConditionalGeneration | ✔️ | ✔️ |
931+
Llava 1.5-7B | LlavaForConditionalGeneration | ✅ | ✅ |
932+
Llava 1.5-13B | LlavaForConditionalGeneration | ✔️ | ✔️ |
930933

931934
**Note**: vLLM currently does not support inference with LoRA-tuned vision models. To use a tuned LoRA adapter of vision model, please merge it with the base model before running vLLM inference.
932935

0 commit comments

Comments
 (0)