[Tests] fix some fast gpu tests.#9379

sayakpaul merged 3 commits intomainhuggingface/diffusers:mainfrom

fast-gpu-test-fixes-iihuggingface/diffusers:fast-gpu-test-fixes-iiCopy head branch name to clipboard