Understanding how data is split between GPUs when using .predict()
#9773
Unanswered
aakaashjois
asked this question in
DDP / multi-GPU / multi-node
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hello,
I have a model trained with PyTorch Lightning and I am trying to use it for making predictions on a set of data. I use the
predict_step
method to handle the necessary work to return an output. I have a callback which,on_predict_batch_end
writes the set of inputs and outputs to a file for each GPU. I am using 8 GPUs withddp
and batch size of 1 to run this whole setup.Looking at the data which is written, I see that all 8 GPUs receive the same inputs. Based on what I understand from Multi-GPU Batch Size docs, each GPU should be receive a different batch of data.
Can anyone help me out on trying to understand how the data is being distributed?
Thank you!
Beta Was this translation helpful? Give feedback.
All reactions