What's the difference between DreamBooth LoRa and traditional LoRa?

[MetaInsight7]

I see a lot of examples using DreamBooth LoRa training code. What's the difference between this and traditional LoRa training? Can this DreamBooth LoRa training code be adapted to standard SFT LoRa code? Does disabling with_prior_preservation return normal LoRa training?

[Kaihua1203]

Yes, I have the same question!

[asomoza]

Hi, are you asking this after reading the Dreambooth paper?

The prior preservation is optional and its used to what the name says, to preserve the knowledge of the model, for example if you're training a special breed of dog, you use prior preservation so not all the generated dogs are the breed you trained, if you don't care about this, which is common, you don't have to use prior preservation.

Dreambooth is a very different target training that normal SFT training, lora or not. Primarily is that it uses a very small dataset and uses instance and class tokens, that's why you sometimes see something like photo of a sks dog. Normal finetuning uses images with normal text descriptions, and because of this, the model needs a significant bigger dataset to learn something because the model is learning everything about the image.

A shorter answer to your questions just in case:

DreamBooth LoRa training code be adapted to standard SFT LoRa code

Yes

Does disabling with_prior_preservation return normal LoRa training?

No

[MetaInsight7]

Thank you very much for your reply.
I only have doubts when I see many open-source Lora training scripts in the dreambooth folder.
I just carefully read through the code and found that if my training set uses regular image text pairs and loads them with -- dataset_name, disabling with_prior_reservation, then the dreambooth_lora training code can become traditional sft lora training, right？

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.