[LoRA] fix dora test to catch the warning properly.#9627

sayakpaul merged 2 commits intomainhuggingface/diffusers:mainfrom

dora-testshuggingface/diffusers:dora-testsCopy head branch name to clipboard