Skip to content

Commit 3985d07

Browse files
committed
fix readme
1 parent 50dff0b commit 3985d07

File tree

2 files changed

+4
-4
lines changed

2 files changed

+4
-4
lines changed

recipes/quickstart/finetuning/datasets/ocrvqa_dataset.py

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,9 +19,7 @@ def replace_target(target,seq):
1919
seq[i],seq[i+1],seq[i+2] = -100,-100,-100
2020
return seq
2121
def tokenize_dialogs(dialogs, images, processor):
22-
# If vocab size is above 128000, use the chat template to generate the tokens as it is from Llama 3 family models
2322
text_prompt = processor.apply_chat_template(dialogs)
24-
#print("text_prompt",text_prompt)
2523
batch = processor(images=images, text=text_prompt,padding = True, return_tensors="pt")
2624
label_list = []
2725
for i in range(len(batch["input_ids"])):

recipes/quickstart/finetuning/finetune_vision_model.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
## Fine-Tuning Meta Llama Multi Modal Models recipe
2-
This recipe steps you through how to finetune a Llama 3.2 vision model on the VQA task using the [OCRVQA](https://huggingface.co/datasets/HuggingFaceM4/the_cauldron/viewer/ocrvqa?row=0) dataset.
2+
This recipe steps you through how to finetune a Llama 3.2 vision model on the OCR VQA task using the [OCRVQA](https://huggingface.co/datasets/HuggingFaceM4/the_cauldron/viewer/ocrvqa?row=0) dataset.
3+
4+
**Disclaimer** As our vision models already have a very good OCR ability, here we just use the OCRVQA dataset to demonstrate the steps needed for fine-tuning our vision models.
35

46
### Fine-tuning steps
57

@@ -20,7 +22,7 @@ For **LoRA finetuning with FSDP**, we can run the following code:
2022

2123
For more details about the finetuning configurations, please read the [finetuning readme](./README.md).
2224

23-
### How to use custom dataset to fine-tune vision model
25+
### How to use a custom dataset to fine-tune vision model
2426

2527
In order to use a custom dataset, please follow the steps below:
2628

0 commit comments

Comments
 (0)