accumulate_grad_batches and DDP

[RaivoKoot]

Hi!
When I use no multi-gpu settings and just a single GPU and the following parameters:

batch_size = 16
accumulate_grad_batches=2

my effective batch size is 32. My question is how accumulate_grad_batches and DDP interact.
If I am using 2 GPUS that are on the same machine and i use the parameters

batch_size = 16
accumulate_grad_batches=2
accelerator='ddp'
gpus=[0,1]

is my effective batch size now 64?
Thanks for the help!

[carmocca]

Yes: https://pytorch-lightning.readthedocs.io/en/latest/advanced/multi_gpu.html#batch-size