[bitsandbytes] improve dtype mismatch handling for bnb + lora.#11270

sayakpaul merged 11 commits intomainhuggingface/diffusers:mainfrom

improve-dtype-mismatch-bnb-lorahuggingface/diffusers:improve-dtype-mismatch-bnb-loraCopy head branch name to clipboard