DP training with low GPU usage

[tonystarkiss]

I trained my model with dp, but I find my GPU SM usage only 50~60%, usually the deep learning training the GPU usage can up to 100%, WHY?

[njzjz]

You can consider increasing the batch size.

[tonystarkiss]

the batch_size is "auto", I do not know when I is set with "auto" how to caculate the batch size? when I set batch size=4, the GPU usage up to 90%.

[njzjz]

Please read the document. You are free to set an atomic number other than 32.