[core] support device type device_maps to work with offloading.#12811

sayakpaul wants to merge 2 commits intomainhuggingface/diffusers:mainfrom

device-map-directhuggingface/diffusers:device-map-directCopy head branch name to clipboard