Fix Group offloading behaviour when using streams#11097

a-r-r-o-w merged 3 commits intomainhuggingface/diffusers:mainfrom

fix-group-offloading-leaf-streamhuggingface/diffusers:fix-group-offloading-leaf-streamCopy head branch name to clipboard