Fix unwrap hooks when the model is wrapped #10730

SunMarc wants to merge 3 commits intomainhuggingface/diffusers:mainfrom

fix-hook-removal-with-warped-modelhuggingface/diffusers:fix-hook-removal-with-warped-modelCopy head branch name to clipboard