VRAM exhaustion during SDXL inference on 8GB AMD GPU (Vulkan backend) causes fragmented VAE processing #761
Description
When running SDXL models using Vulkan backend on an AMD Radeon RX 6600 (8GB VRAM), the UNET computation exhausts nearly all available VRAM. This forces subsequent VAE decoding to operate in fragmented slices.
Optimization Suggestions:
Model Swapping with Caching
During batch image generation:
-
Cache intermediate latent tensors after UNET computation
-
Completely unload the UNET model from VRAM
-
Load VAE model once and process all cached latents collectively
Asynchronous CPU Offloading
Implement pipelining where:
-
GPU processes UNET for image N+1
-
CPU simultaneously decodes VAE for image N (using cached latents)
These approaches could significantly reduce VRAM pressure and potentially improve throughput on GPUs with limited memory.