About GPU type and batch size

[leo-frank]

It was noted in paper that your method was trained "approximately 30 minutes of finetuning on a single A100 GPU"

I am curious about two questions:

1. GPU meomery: 40G or 80G
2. batchsize is 4 or any other number

[stefan-baumann]

Hi,

We used a batch size of 8 (which is effectively increased by a factor of 3 due to the stratification), as noted in Sec. 4.1.

We generally used 80GB A100s, but this is only because our internal cluster does not have 40GB A100s. I can't tell you off the top of my head what the exact tight VRAM requirements for our training setup are, but you should be easily able to perform, e.g., SD2.1 training on an A100 40GB. If it should not fit initially, gradient accumulation should alleviate that without losing (much) training efficiency.

[leo-frank]

I am also curious about the training iterations.
It's said that you trained for 400 iterations as noted in paper.

1. So, it means that only 800 images are used in your training. ( as the batch contains two diffrent images, each with 3 timestep).
   My understanding is correct or not ?

2. I wonder the performance if we train longer ? have you test that ? will the performance drops?

[stefan-baumann]

It's 3200 images, 400 * 8. We don't see any improvements after ~400 steps.