Skip to content

Commit 2edeb63

Browse files
author
Yang Yang(Tony)
authored
Update parallel_do.md (#8665)
1 parent 718642e commit 2edeb63

File tree

1 file changed

+11
-10
lines changed

1 file changed

+11
-10
lines changed

doc/design/parallel_do.md

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -39,15 +39,16 @@ In the backward pass
3939
This implementation allows to write mixed device program like this
4040
4141
```python
42-
# get embedding feature on CPU
43-
feature = some_cpu_only_op(data)
42+
W1 = fluid.tensor(size=[100,20], parameter=true)
43+
W2 = fluid.tensor(size=[20,15], parameter=true)
4444
45-
gpu_places = get_place(use_gpu=True)
45+
data = layers.data()
46+
47+
gpu_places = layers.get_place(use_gpu=True)
4648
# parallel processing on multiple GPUs
4749
pd = ParallelDo(gpu_places)
48-
with pd.do():
49-
read_input(feature)
50-
prediction = my_net(feature)
50+
with pd.do(input=data):
51+
prediction = softmax(fc(fc(data, W1), W2))
5152
write_output(prediction)
5253
prediction = pd()
5354
loss = cross_entropy(prediction, label)
@@ -66,20 +67,20 @@ start_program
6667
main_program
6768
{
6869
block0 {
69-
vars: data, places, w1, w2
70+
vars: data, places, w1, w2, w1_grad, w2_grad,
7071
ops: data, get_place, parallel_do(block1),
7172
parallel_do_grad(block2),
7273
sgd(w2, w2_grad),
7374
sgd(w1, w1_grad)
7475
}
75-
block1 {
76+
block1 { # the forward pass
7677
parent_block: 0
7778
vars: data, h1, h2, loss
7879
ops: fc, fc, softmax
7980
}
80-
block2 {
81+
block2 { # the backward pass
8182
parent_block: 1
82-
vars: data_grad, h1_grad, h2_grad, loss_gard, w1_grad, w2_grad
83+
vars: data_grad, h1_grad, h2_grad, loss_gard, local_w1_grad, local_w2_grad
8384
ops: softmax_grad,
8485
fc_grad
8586
fc_grad

0 commit comments

Comments
 (0)