1
1
mpi4py
2
2
======
3
3
4
- MPI which stands for message passing interface is a common library
5
- for parallel programming. There is a package mpi4py that builds on
6
- the top of mpi, and lets arbitrary python objects be passed between
7
- different processes. These packages are not part of the default
8
- sage install. To install them do
4
+ MPI, which stands for Message Passing Interface, is a common library
5
+ for parallel programming. There is a package ``mpi4py `` that builds on
6
+ the top of MPI, and lets arbitrary python objects be passed between
7
+ different processes. These packages are not available from the
8
+ Sage distribution. Install ``openmpi `` using your distribution's
9
+ package manager. Then install ``mpi4py `` using
9
10
10
11
.. skip
11
12
12
13
::
13
14
14
- sage: optional_packages()
15
+ sage: !pip install mpi4py
15
16
16
- Find the package name openmpi-\* and mpi4py-\* and do
17
-
18
- .. skip
19
-
20
- ::
21
-
22
- sage: install_package('openmpi-*')
23
- sage: install_package('mpi4py-*')
24
-
25
- Note that openmpi takes a while to compile (15-20 minutes or so).
26
- Openmpi can be run on a cluster, however this requires some set up
27
- so that processes on different machines can communicate (though if
28
- you are on a cluster this is probably already set up). The simplest
29
- case is if you are on a shared memory or multicore system where
30
- openmpi will just work with no configuration from you. To be
31
- honest, I have never tried to run mpi4py on a cluster, though there
32
- is much information about these topics online.
33
-
34
- Now, the way that mpi works is you start a group of mpi processes,
17
+ Now, the way that MPI works is you start a group of MPI processes,
35
18
all of the processes run the same code. Each process has a rank,
36
19
that is a number that identifies it. The following pseudocode
37
20
indicates the general format of MPI programs.
@@ -48,163 +31,159 @@ indicates the general format of MPI programs.
48
31
else if my rank is n+1:
49
32
....
50
33
51
- Each processes looks for what it's supposed to do (specified by its
52
- rank) and processes can send data and receive data. Lets give an
53
- example. Create a script with the following code in a file mpi_1.py
34
+ Each process looks for what it's supposed to do (specified by its
35
+ rank), and processes can send data and receive data. Let's give an
36
+ example. Create a script with the following code in a file `` mpi_1.py ``
54
37
55
38
.. CODE-BLOCK :: python
56
39
57
40
from mpi4py import MPI
58
41
comm = MPI .COMM_WORLD
59
42
print (" hello world" )
60
- print (" my rank is: %d " % comm.rank)
43
+ print (f " my rank is: { comm.rank} " )
61
44
62
- To run it you can do (from the command line in your sage
45
+ To run it you can do (from the command line in your Sage
63
46
directory)
64
47
65
48
.. CODE-BLOCK :: shell-session
66
49
67
- ./local/bin/mpirun -np 5 ./sage -python mpi_1.py
50
+ mpirun -np 5 ./sage -python mpi_1.py
51
+
52
+ The command ``mpirun -np 5 `` starts 5 copies of a program under MPI. In
53
+ this case we have 5 copies of Sage in pure Python mode running the
54
+ script ``mpi_1.py ``. The result should be 5 "hello worlds" plus 5 distinct ranks.
68
55
69
- The command mpirun -np 5 starts 5 copies of a program under mpi. In
70
- this case we have 5 copies of sage in pure python mode run the
71
- script mpi_1.py. The result should be 5 "hello worlds" plus 5 distinct ranks.
72
- The two most important mpi operations are sending and receiving.
73
- Consider the following example which you should put in a script mpi_2.py
56
+ The two most important MPI operations are sending and receiving.
57
+ Consider the following example which you should put in a script ``mpi_2.py ``
74
58
75
59
.. CODE-BLOCK :: python
76
60
77
61
from mpi4py import MPI
78
62
import numpy
79
63
comm = MPI .COMM_WORLD
80
- rank= comm.rank
81
- size= comm.size
82
- v= numpy.array([rank]* 5 , dtype = float )
83
- comm.send(v,dest = (rank+ 1 )% size)
84
- data= comm.recv(source = (rank- 1 )% size)
85
- print (" my rank is %d " % rank)
64
+ rank = comm.rank
65
+ size = comm.size
66
+ v = numpy.array([rank] * 5 , dtype = float )
67
+ comm.send(v, dest = (rank+ 1 ) % size)
68
+ data = comm.recv(source = (rank- 1 ) % size)
69
+ print (f " my rank is: { rank} " )
86
70
print (" I received this:" )
87
71
print (data)
88
72
89
- The same command as above with mpi_1.py replaced by mpi_2.py will
90
- produce 5 outputs and you will see each process creates an array and
91
- then passes it to the next guy ( where the last guy passes to the
92
- first.) Note that MPI.size is the total number of mpi
93
- processes. MPI.COMM WORLD is the communication world.
73
+ The same command as above with `` mpi_1.py `` replaced by `` mpi_2.py `` will
74
+ produce 5 outputs. Each process will create an array and pass
75
+ it to the next process, where the last process passes to the
76
+ first. Note that `` MPI.size `` is the total number of MPI
77
+ processes. `` MPI.COMM_WORLD `` is the communication world.
94
78
95
79
There are some subtleties regarding MPI to be aware of. Small sends
96
80
are buffered. This means if a process sends a small object it will
97
81
be stored by openmpi and that process will continue its execution
98
82
and the object it sent will be received whenever the destination
99
- executes a receive. However, if an object is large a process will
83
+ executes a receive. However, if an object is large, a process will
100
84
hang until its destination executes a corresponding receive. In
101
- fact the above code will hang if [rank]\* 5 is replaced by
102
- [rank]\ * 500. It would be better to do
85
+ fact, the above code will hang if `` [rank]*5 `` is replaced by
86
+ `` [rank]*500 `` . It would be better to do
103
87
104
88
.. CODE-BLOCK :: python
105
89
106
90
from mpi4py import MPI
107
91
import numpy
108
92
comm = MPI .COMM_WORLD
109
- rank= comm.rank
110
- size= comm.size
111
- v= numpy.array([rank]* 500 ,dtype = float )
112
- if comm.rank== 0 :
113
- comm.send(v,dest = (rank+ 1 )% size)
93
+ rank = comm.rank
94
+ size = comm.size
95
+ v = numpy.array([rank] * 500 , dtype = float )
96
+ if comm.rank == 0 :
97
+ comm.send(v, dest = (rank+ 1 ) % size)
114
98
if comm.rank > 0 :
115
- data= comm.recv(source = (rank- 1 )% size)
116
- comm.send(v,dest = (rank+ 1 )% size)
117
- if comm.rank== 0 :
118
- data= comm.recv(source = size- 1 )
99
+ data = comm.recv(source = (rank- 1 ) % size)
100
+ comm.send(v, dest = (rank+ 1 ) % size)
101
+ if comm.rank == 0 :
102
+ data = comm.recv(source = size - 1 )
119
103
120
- print (" my rank is %d " % rank)
104
+ print (f " my rank is: { rank} " )
121
105
print (" I received this:" )
122
106
print (data)
123
107
124
- Now the first process initiates a send, and then process 1 will be
125
- ready to receive and then he will send and process 2 will be
126
- waiting to receive, etc. This will not lock regardless of how large
127
- of an array we pass .
108
+ Now, process 0 sends the data to process 1, then waits to receive from
109
+ process `` MPI.size - 1 ``. Simultaneously, process 1 will send the
110
+ data to process 2, then receives the data from process 0. This will
111
+ not lock even if the array transmitted is huge .
128
112
129
- A common idiom is to have one process, usually the one with rank 0
130
- act as a leader. That processes sends data out to the other
131
- processes and processes the results and decides how further
113
+ A common idiom is to have one process, usually the one with rank 0,
114
+ act as a leader. That process sends data out to the other
115
+ processes, compute on the results, and decides how much further
132
116
computation should proceed. Consider the following code
133
117
134
118
.. CODE-BLOCK :: python
135
119
136
120
from mpi4py import MPI
137
121
import numpy
138
- sendbuf= []
139
- root= 0
122
+ sendbuf = []
123
+ root = 0
140
124
comm = MPI .COMM_WORLD
141
- if comm.rank== 0 :
142
- m= numpy.random.randn(comm.size,comm.size)
125
+ if comm.rank == 0 :
126
+ m = numpy.random.randn(comm.size, comm.size)
143
127
print (m)
144
128
sendbuf= m
145
129
146
- v= comm.scatter(sendbuf,root)
130
+ v = comm.scatter(sendbuf, root)
147
131
148
132
print (" I got this array:" )
149
133
print (v)
150
134
151
- The scatter command takes a list and evenly divides it amongst all
135
+ The `` scatter `` command takes a list and evenly divides it amongst all
152
136
the processes. Here the root process creates a matrix (which is
153
- viewed as a list of rows) and then scatters it to everybody (roots
154
- sendbuf is divided equally amongst the processes). Each process
155
- prints the row it got. Note that the scatter command is executed by
156
- everyone, but when root executes it, it acts as a send and a
157
- receive (root gets one row from itself), while for everyone else it
158
- is just a receive.
159
-
160
- There is a complementary gather command that collects results from
161
- all the processes into a list. The next example uses scatter and
162
- gather together. Now the root process scatters the rows of a
163
- matrix, each process then squares the elements of the row it gets.
164
- Then the rows are all gathered up again by the root process who
165
- collects them into a new matrix.
137
+ viewed as a list of rows) and then scatters it to everybody (root's
138
+ ``sendbuf `` is divided equally amongst the processes). Each process
139
+ prints the row it got. Note that the ``scatter `` command is executed by
140
+ everyone, but when root executes it, it acts as a ``send `` and a
141
+ ``receive `` (root gets one row from itself), while for everyone else it
142
+ is just a ``receive ``.
143
+
144
+ There is a complementary ``gather `` command that collects results from
145
+ all the processes into a list. The next example uses ``scatter `` and
146
+ ``gather `` together. Now the root process scatters the rows of a
147
+ matrix. Each process squares the elements of the row it receives.
148
+ The root process then gathers the rows into a new matrix.
166
149
167
150
.. CODE-BLOCK :: python
168
151
169
152
from mpi4py import MPI
170
153
import numpy
171
154
comm = MPI .COMM_WORLD
172
- sendbuf= []
173
- root= 0
174
- if comm.rank== 0 :
175
- m= numpy.array(range (comm.size* comm.size),dtype = float )
176
- m.shape= (comm.size,comm.size)
155
+ sendbuf = []
156
+ root = 0
157
+ if comm.rank == 0 :
158
+ m = numpy.array(range (comm.size * comm.size), dtype = float )
159
+ m.shape = (comm.size, comm.size)
177
160
print (m)
178
- sendbuf= m
161
+ sendbuf = m
179
162
180
- v= comm.scatter(sendbuf,root)
163
+ v = comm.scatter(sendbuf, root)
181
164
print (" I got this array:" )
182
165
print (v)
183
- v= v* v
184
- recvbuf= comm.gather(v,root)
185
- if comm.rank== 0 :
166
+ v = v* v
167
+ recvbuf = comm.gather(v, root)
168
+ if comm.rank == 0 :
186
169
print (numpy.array(recvbuf))
187
170
188
- There is also a broadcast command that sends a single object to
171
+ There is also a `` broadcast `` command that sends a single object to
189
172
every process. Consider the following small extension. This is the
190
- same as before, but now at the end the root process sends everyone
173
+ same as before, but now at the end, the root process sends everyone
191
174
the string "done", which is printed out.
192
175
193
176
.. CODE-BLOCK :: python
194
177
195
- v= MPI .COMM_WORLD .scatter(sendbuf,root)
178
+ v = MPI .COMM_WORLD .scatter(sendbuf, root)
196
179
print (" I got this array:" )
197
180
print (v)
198
- v= v* v
199
- recvbuf= MPI .COMM_WORLD .gather(v,root)
200
- if MPI .COMM_WORLD .rank== 0 :
181
+ v = v* v
182
+ recvbuf = MPI .COMM_WORLD .gather(v, root)
183
+ if MPI .COMM_WORLD .rank == 0 :
201
184
print (numpy.array(recvbuf))
202
185
203
- if MPI .COMM_WORLD .rank== 0 :
204
- sendbuf= " done"
205
- recvbuf= MPI .COMM_WORLD .bcast(sendbuf,root)
186
+ if MPI .COMM_WORLD .rank == 0 :
187
+ sendbuf = " done"
188
+ recvbuf = MPI .COMM_WORLD .bcast(sendbuf,root)
206
189
print (recvbuf)
207
-
208
- MPI programming is difficult. It is "schizophrenic programming" in
209
- that you are writing a single programming with multiple threads of
210
- execution "many voices in one head".
0 commit comments