fix: restore module buffers during stream-based group offload#13238

Chase-Xuu wants to merge 1 commit intohuggingface:mainhuggingface/diffusers:mainfrom

Chase-Xuu:fix/group-offload-missing-module-buffersChase-Xuu/diffusers:fix/group-offload-missing-module-buffersCopy head branch name to clipboard