Revert "[LoRA] fix: lora loading when using with a device_mapped mode…#9823

yiyixuxu merged 1 commit intomainhuggingface/diffusers:mainfrom

reverthuggingface/diffusers:revertCopy head branch name to clipboard