-
|
I just found out that Prodigy Adv is using much lower LR for OFTv2 training. Is this normal? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 3 replies
-
|
Yes, OFTv2 should be used with a lower LR than a normal lora. |
Beta Was this translation helpful? Give feedback.
-
|
Do you mean Prodigy Adv or normal Prodigy? |
Beta Was this translation helpful? Give feedback.
-
|
Can I ask why you think there is a difference? prodigy advanced has options that definitely don’t work with oft I use prodigy schedule free myself, and it’s been ok with oft but it’s much slower when using schedule free |
Beta Was this translation helpful? Give feedback.
-
|
oh you mean Prodigy in general |
Beta Was this translation helpful? Give feedback.
Yes, OFTv2 should be used with a lower LR than a normal lora.
Prodigy, in general, does not work well with OFTv2. Many people have reported that Prodigy struggles to find a LR for OFTv2 runs.
Normally you do not train the text encoder with Prodigy, unless you can use split groups, the text encoder in general likes to have a very low LR.
Whether OFTv2 follows historical info regarding TE training is an open discussion I suppose.