Explain collator in SFTTrainer

[konstantinos-p]

Explain the role of the SFTTrainer in inferring that the model is a vision-language model and applying the appropriate collator.

What does this PR do?

I was looking for resources for finetuning VLMs, specifically Qwen models. I followed the Qwen2-VL-7B model card and then the fine_tuning_vlm_trl.ipynb notebook. In both cases it is clear how to do inference and the preprocessing is done explicitly. However, when doing training the preprocessing is done implicitly through a collator that is infered by the trainer. This is not explained in the current notebook and took me some time to figure out especially since the notebook is resource heavy. I added a short passage in section 4.3 to explain this point and save time for others.

New text:

When doing inference we defined our own generate_text_from_sample function which applied the necessary preprocessing before passing the inputs to the model. Here, the SFTTrainer infers automatically that the model is a vision-language model and applies a DataCollatorForVisionLanguageModeling which convers the inputs to the appropriate format.

Who can review?

@merveenoyan @stevhliu

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[stevhliu]

Thanks for the clarification! Also pinging @sergiopaniego for a look who authored this notebook :)

[sergiopaniego]

Thanks for the addition! This is indeed a super recent feature huggingface/trl#3862 added to TRL.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.