[QwenImage] Do not use `lru_cache` when compiling#12211

cbensimon wants to merge 3 commits intomainhuggingface/diffusers:mainfrom

qwenimage-lru-cache-bypasshuggingface/diffusers:qwenimage-lru-cache-bypassCopy head branch name to clipboard