You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+38-2Lines changed: 38 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -33,6 +33,25 @@ julia> xrange,yrange,zrange = 1:3,2:4,3:6 # ranges should be strictly increasing
33
33
34
34
There are a total of 36 possible `(x,y,z)` combinations possible given these ranges. Let's say that we would like to split the evaluation of the function over 10 processors. We describe the simple way to evaluate this and then explain how this is achieved.
35
35
36
+
The set of parameters may be split up using the function `evenlyscatterproduct`. In this example each of the 10 processors receive a chunk as listed below
37
+
38
+
```julia
39
+
julia> [collect(evenlyscatterproduct((xrange,yrange,zrange),10,i)) for i=1:10]
The first six processors receive 4 tuples of parameters each and the final four receive 3 each. This is the splitting used by the various functions described next.
54
+
36
55
## pmap-related functions
37
56
38
57
The package provides versions of `pmap` with an optional reduction. These differ from the one provided by `Distributed` in a few key aspects: firstly, the iterator product of the argument is what is passed to the function and not the arguments by elementwise, so the i-th task will be `Iterator.product(args...)[i]` and not `[x[i] for x in args]`. Specifically the second set of parameters in the example above will be `(2,2,3)` and not `(2,3,4)`.
The functions `pmapreduce` produces the same result as `pmapreduce_commutative` if the reduction operator is commutative (ie. the order of results received from the children workers does not matter). The function `pmapreduce_commutative` might be faster as it avoids an intermediate collection and sorting. This is what is used by the function `pmapsum` that chooses the reduction operator to be a sum.
139
+
140
+
```julia
141
+
julia>sum(workers())
142
+
5
143
+
144
+
# We compute ones(2).*sum(workers()) in parallel
145
+
julia>pmapsum(x->ones(2).*myid(),1:nworkers())
146
+
2-element Array{Float64,1}:
147
+
5.0
148
+
5.0
149
+
```
150
+
115
151
## ProductSplit
116
152
117
153
In the above examples we have talked about the tasks being distributed approximately equally among the workers without going into details about the distribution, which is what we describe here. The package provides an iterator `ProductSplit` that lists that ranges of parameters that would be passed on to each core. This may equivalently be achieved using an
with appropriately chosen parameters, and in many ways a `ProductSplit` behaves similarly. However a `ProductSplit` supports several extra features such as `O(1)` indexing, which eliminates the need to actually iterate over it in many scenarios.
0 commit comments