Skip to content

Commit ec3f4fd

Browse files
committed
removed vestigial throwRemoteException, updated Readme
1 parent 616aa61 commit ec3f4fd

File tree

2 files changed

+38
-7
lines changed

2 files changed

+38
-7
lines changed

README.md

Lines changed: 38 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,25 @@ julia> xrange,yrange,zrange = 1:3,2:4,3:6 # ranges should be strictly increasing
3333

3434
There are a total of 36 possible `(x,y,z)` combinations possible given these ranges. Let's say that we would like to split the evaluation of the function over 10 processors. We describe the simple way to evaluate this and then explain how this is achieved.
3535

36+
The set of parameters may be split up using the function `evenlyscatterproduct`. In this example each of the 10 processors receive a chunk as listed below
37+
38+
```julia
39+
julia> [collect(evenlyscatterproduct((xrange,yrange,zrange),10,i)) for i=1:10]
40+
10-element Array{Array{Tuple{Int64,Int64,Int64},1},1}:
41+
[(1, 2, 3), (2, 2, 3), (3, 2, 3), (1, 3, 3)]
42+
[(2, 3, 3), (3, 3, 3), (1, 4, 3), (2, 4, 3)]
43+
[(3, 4, 3), (1, 2, 4), (2, 2, 4), (3, 2, 4)]
44+
[(1, 3, 4), (2, 3, 4), (3, 3, 4), (1, 4, 4)]
45+
[(2, 4, 4), (3, 4, 4), (1, 2, 5), (2, 2, 5)]
46+
[(3, 2, 5), (1, 3, 5), (2, 3, 5), (3, 3, 5)]
47+
[(1, 4, 5), (2, 4, 5), (3, 4, 5)]
48+
[(1, 2, 6), (2, 2, 6), (3, 2, 6)]
49+
[(1, 3, 6), (2, 3, 6), (3, 3, 6)]
50+
[(1, 4, 6), (2, 4, 6), (3, 4, 6)]
51+
```
52+
53+
The first six processors receive 4 tuples of parameters each and the final four receive 3 each. This is the splitting used by the various functions described next.
54+
3655
## pmap-related functions
3756

3857
The package provides versions of `pmap` with an optional reduction. These differ from the one provided by `Distributed` in a few key aspects: firstly, the iterator product of the argument is what is passed to the function and not the arguments by elementwise, so the i-th task will be `Iterator.product(args...)[i]` and not `[x[i] for x in args]`. Specifically the second set of parameters in the example above will be `(2,2,3)` and not `(2,3,4)`.
@@ -104,19 +123,36 @@ julia> pmapreduce_commutative_elementwise(x->x^2,prod,1:3)
104123
The function `pmapreduce` sorts the results obtained from each processor, so it is useful for concatenations.
105124

106125
```julia
107-
# We perform the job on 2 workers
126+
julia> workers()
127+
2-element Array{Int64,1}:
128+
2
129+
3
130+
108131
# The signature is pmapreduce(fmap,freduce,iterable)
109132
julia> pmapreduce(x->ones(2).*myid(),x->hcat(x...),1:nworkers())
110133
2×2 Array{Float64,2}:
111134
2.0 3.0
112135
2.0 3.0
113136
```
114137

138+
The functions `pmapreduce` produces the same result as `pmapreduce_commutative` if the reduction operator is commutative (ie. the order of results received from the children workers does not matter). The function `pmapreduce_commutative` might be faster as it avoids an intermediate collection and sorting. This is what is used by the function `pmapsum` that chooses the reduction operator to be a sum.
139+
140+
```julia
141+
julia> sum(workers())
142+
5
143+
144+
# We compute ones(2).*sum(workers()) in parallel
145+
julia> pmapsum(x->ones(2).*myid(),1:nworkers())
146+
2-element Array{Float64,1}:
147+
5.0
148+
5.0
149+
```
150+
115151
## ProductSplit
116152

117153
In the above examples we have talked about the tasks being distributed approximately equally among the workers without going into details about the distribution, which is what we describe here. The package provides an iterator `ProductSplit` that lists that ranges of parameters that would be passed on to each core. This may equivalently be achieved using an
118154

119-
```Iterator.Take{Iterator.Drop{Iterator.ProductIterator}}```
155+
```Iterators.Take{Iterators.Drop{Iterators.ProductIterator}}```
120156

121157
with appropriately chosen parameters, and in many ways a `ProductSplit` behaves similarly. However a `ProductSplit` supports several extra features such as `O(1)` indexing, which eliminates the need to actually iterate over it in many scenarios.
122158

src/ParallelUtilities.jl

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -543,11 +543,6 @@ nprocs_node(procs_used::Vector{<:Integer} = workers()) = nprocs_node(gethostname
543543
# pmapsum and pmapreduce
544544
############################################################################################
545545

546-
function throwRemoteException(e::Exception)
547-
c = CapturedException(e,catch_backtrace())
548-
throw(RemoteException(c))
549-
end
550-
551546
# This function does not sort the values, so it might be faster
552547
function pmapreduce_commutative(fmap::Function,freduce::Function,iterators::Tuple,args...;kwargs...)
553548

0 commit comments

Comments
 (0)