adjust to get CI test cases passed on XPU#11759

a-r-r-o-w merged 8 commits intohuggingface:mainhuggingface/diffusers:mainfrom

kaixuanliu:xpu-precision-utkaixuanliu/diffusers:xpu-precision-utCopy head branch name to clipboard