[CI] uninstall deps properly from pr gpu tests.#11102

sayakpaul merged 3 commits intomainhuggingface/diffusers:mainfrom

uninstall-properly-pr-gpu-testshuggingface/diffusers:uninstall-properly-pr-gpu-testsCopy head branch name to clipboard