Fix compatibility with pipeline when loading model with device_map on single gpu#10390

SunMarc merged 2 commits intomainhuggingface/diffusers:mainfrom

remove-force-hook-from-model-inithuggingface/diffusers:remove-force-hook-from-model-initCopy head branch name to clipboard