Memory of gradient calculation distributed across GPUs? #19393
Unanswered
rkruegs123
asked this question in
Q&A
Replies: 1 comment
-
|
Just bumping this @mattjj |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I have a question about how the gradient calculation can be distributed across GPUs.
Consider a calculation in which there is
pmap(orshmap/xmap) inside an outer scan. See #1369 for an example. Then, consider taking the gradient of this calculation. Does the gradient calculation get distributed across devices?Beta Was this translation helpful? Give feedback.
All reactions