Why isnt my GPU generate images in parallel, given batch size >1

[lxrswdd]

I am running on an A6000 GPU.
I am using juggernautXL_v8Rundiffusion, Version: v1.7.0. so it is an SDXL model.
The record below suggests that stable diffusion only used up 12 GB VRAM and the time taken is increased tripled.
I would expect the model to generate 3 images using 15 seconds as long as the VRAM can handle it.

When batch size =1:
Time taken: 15.5 sec.
A: 9.25 GB, R: 11.80 GB, Sys: 12.2/47.5361 GB (25.6%)

When Batch size = 3:
Time taken: 44.9 sec.
A: 9.88 GB, R: 11.60 GB, Sys: 12.0/47.5361 GB (25.2%)

Also, the processing speed is halved when running batch size=3.
It is around 2.3 s/it for batch size = 1, but only 1.2 for batch size = 3

[w-e-w]

it is processing the image in parallel but the performance is capped by GPU Core not VRAM

[lxrswdd]

Do you mean the number of CUDA cores? I mean even the H100 only has about 15000 of them and A6000 has about 10000. Does it mean it is unlikely for us to process the images in parallel using the existing GPUs? Does Automatic1111 support multi-gpu generation tasks?

[w-e-w]

stating the obvious
processing something in parallel does that mean performance benefit
you only get performance benefit from parallelizing when if parallelizing removes the bottleneck

webui IS processing the images in parallel
but that doesn't mean that you will see a performance difference

---

extremely simplified
batch size increases the number of work that your processor is requested to do in a given time
for example
if at batch size of 1 processor utilization is only 50% that it has spare room processing more work in parallel
now if you said batch size 2 and it's able to utilize the processor to 100%, it will be able to do the doble the ammount work in the same amount of time
but now if you increase the batch size to 4, you're basically asking the same processor to do 200% of the work that it's able to do,
since it's not able to do so it's going to take double the amount of time to complete the job

when generating "normal size" images the bottleneck is the GPU processing unit
which means giving it more work to do at the same time is not going to improve performance as it does not remove the bottleneck

---

if you want to see the performance difference between high batch size vs high batch count
generate vary small images at resolution pf 64x64

yes the output will be ugly but this is not the point

batch count 8	batch size 8

this is on a 3090
you can see when generating small images it almost scales linearly
batch count of 8 took 45.4 seconds
batch size of 8 took only 5.9 seconds, 7.6 times faster

---

Does Automatic1111 support multi-gpu generation tasks

no