Finetuning Generative Q&A Model (e.g. google/flan-t5-base)

[demongolem-biz2]

I have gone through the example for finetuning using distilibert and that all worked as it should have. However, when we turn to Generative Q&A and flan is used by default, we can't use that same process for finetuning. If I had Question and Answers from a very specific domain, how would I go about finetuning with Generative Q&A and how is it different than https://haystack.deepset.ai/tutorials/02_finetune_a_model_on_your_data?

[julian-risch]

@demongolem-biz2 Finetuning generative models is something we currently do not support in Haystack. I'd recommend to have a look at the Seq2SeqTrainer class in HuggingFace transformers if you really want to finetune such a model. However, I wouldn't recommend it right away because it is really a lot of effort. Even the dataset preparation will take some time plus hyperparameters of the training, and so on.
Instead, I'd suggest to have a look at RAG pipelines in Haystack, for example here: https://haystack.deepset.ai/tutorials/07_rag_generator and more advanced details here: https://haystack.deepset.ai/blog/enhancing-rag-pipelines-in-haystack