[ci] enable hotswapping tests on our nightly CI.#11826

sayakpaul wants to merge 2 commits intomainhuggingface/diffusers:mainfrom

fix-hotswapping-cihuggingface/diffusers:fix-hotswapping-ciCopy head branch name to clipboard