Skip to content

Commit 74a62f0

Browse files
twitwitwitwi
committed
[DOI: 10.5281/zenodo.57492] Rebuilt HTML files for release 2016.06
jekyll version: jekyll 3.1.6
1 parent 778a7d4 commit 74a62f0

File tree

21 files changed

+11074
-0
lines changed

21 files changed

+11074
-0
lines changed

‎01-numpy/index.html‎

Lines changed: 1252 additions & 0 deletions
Large diffs are not rendered by default.

‎02-loop/index.html‎

Lines changed: 507 additions & 0 deletions
Large diffs are not rendered by default.

‎03-lists/index.html‎

Lines changed: 649 additions & 0 deletions
Large diffs are not rendered by default.

‎04-files/index.html‎

Lines changed: 356 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,356 @@
1+
<!doctype html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="utf-8">
5+
<meta http-equiv="X-UA-Compatible" content="IE=edge">
6+
<meta http-equiv="last-modified" content="2016-07-19 12:35:03 +0200">
7+
<meta name="viewport" content="width=device-width, initial-scale=1">
8+
<meta name="search-domain" value="/swc-releases/2016.06/python-novice-inflammation">
9+
<link rel="stylesheet" type="text/css" href="/swc-releases/2016.06/python-novice-inflammation/assets/css/bootstrap.css" />
10+
<link rel="stylesheet" type="text/css" href="/swc-releases/2016.06/python-novice-inflammation/assets/css/bootstrap-theme.css" />
11+
<link rel="stylesheet" type="text/css" href="/swc-releases/2016.06/python-novice-inflammation/assets/css/lesson.css" />
12+
13+
<link rel="shortcut icon" type="image/x-icon" href="/favicon-swc.ico" />
14+
15+
16+
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
17+
<!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
18+
<!--[if lt IE 9]>
19+
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
20+
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
21+
<![endif]-->
22+
<title>Programming with Python: Analyzing Data from Multiple Files</title>
23+
</head>
24+
<body>
25+
<div class="container">
26+
<nav class="navbar navbar-default">
27+
<div class="container-fluid">
28+
<div class="navbar-header">
29+
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1" aria-expanded="false">
30+
<span class="sr-only">Toggle navigation</span>
31+
<span class="icon-bar"></span>
32+
<span class="icon-bar"></span>
33+
<span class="icon-bar"></span>
34+
</button>
35+
36+
37+
38+
<a href="https://software-carpentry.org" class="pull-left">
39+
<img class="navbar-logo" src="/swc-releases/2016.06/python-novice-inflammation/assets/img/swc-icon-blue.svg" alt="Software Carpentry logo" />
40+
</a>
41+
42+
43+
44+
<a class="navbar-brand" href="/swc-releases/2016.06/python-novice-inflammation/">Home</a>
45+
46+
</div>
47+
<div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
48+
<ul class="nav navbar-nav">
49+
50+
51+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/conduct/">Code of Conduct</a></li>
52+
53+
54+
55+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/setup/">Setup</a></li>
56+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/reference/">Reference</a></li>
57+
<li class="dropdown">
58+
<a href="/swc-releases/2016.06/python-novice-inflammation/" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Episodes <span class="caret"></span></a>
59+
<ul class="dropdown-menu">
60+
61+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/01-numpy/">Analyzing Patient Data</a></li>
62+
63+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/02-loop/">Repeating Actions with Loops</a></li>
64+
65+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/03-lists/">Storing Multiple Values in Lists</a></li>
66+
67+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/04-files/">Analyzing Data from Multiple Files</a></li>
68+
69+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/05-cond/">Making Choices</a></li>
70+
71+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/06-func/">Creating Functions</a></li>
72+
73+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/07-errors/">Errors and Exceptions</a></li>
74+
75+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/08-defensive/">Defensive Programming</a></li>
76+
77+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/09-debugging/">Debugging</a></li>
78+
79+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/10-cmdline/">Command-Line Programs</a></li>
80+
81+
</ul>
82+
</li>
83+
84+
85+
86+
87+
<li class="dropdown">
88+
<a href="/swc-releases/2016.06/python-novice-inflammation/" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Extras <span class="caret"></span></a>
89+
<ul class="dropdown-menu">
90+
91+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/about/">About</a></li>
92+
93+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/discuss/">Discussion</a></li>
94+
95+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/figures/">Figures</a></li>
96+
97+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/extras/guide.html">Programming with Python</a></li>
98+
99+
</ul>
100+
</li>
101+
102+
103+
104+
<li><a href="/swc-releases/2016.06/python-novice-inflammation/license/">License</a></li>
105+
</ul>
106+
<form class="navbar-form navbar-right" role="search" id="search" onsubmit="google_search(); return false;">
107+
<div class="form-group">
108+
<input type="text" id="google-search" placeholder="Search...">
109+
</div>
110+
</form>
111+
</div>
112+
</div>
113+
</nav>
114+
115+
116+
117+
118+
119+
<div class="row">
120+
<div class="col-md-1">
121+
<h3>
122+
123+
<a href="/swc-releases/2016.06/python-novice-inflammation/03-lists/"><span class="glyphicon glyphicon-menu-left"></span></a>
124+
125+
</h3>
126+
</div>
127+
<div class="col-md-10">
128+
<h3 class="maintitle"><a href="/swc-releases/2016.06/python-novice-inflammation/">Programming with Python</a></h3>
129+
<h1 class="maintitle">Analyzing Data from Multiple Files</h1>
130+
</div>
131+
<div class="col-md-1">
132+
<h3>
133+
134+
<a href="/swc-releases/2016.06/python-novice-inflammation/05-cond/"><span class="glyphicon glyphicon-menu-right"></span></a>
135+
136+
</h3>
137+
</div>
138+
</div>
139+
140+
141+
<blockquote class="objectives">
142+
<h2>Overview</h2>
143+
144+
<div class="row">
145+
<div class="col-md-3">
146+
<strong>Teaching:</strong> 20 min
147+
</br>
148+
<strong>Exercises:</strong> 0 min
149+
</div>
150+
<div class="col-md-9">
151+
<strong>Questions</strong>
152+
<ul>
153+
154+
<li><p>How can I do the same operations on many different files?</p>
155+
</li>
156+
157+
</ul>
158+
</div>
159+
</div>
160+
161+
<div class="row">
162+
<div class="col-md-3">
163+
</div>
164+
<div class="col-md-9">
165+
<strong>Objectives</strong>
166+
<ul>
167+
168+
<li><p>Use a library function to get a list of filenames that match a simple wildcard pattern.</p>
169+
</li>
170+
171+
<li><p>Use a for loop to process multiple files.</p>
172+
</li>
173+
174+
</ul>
175+
</div>
176+
</div>
177+
178+
</blockquote>
179+
180+
<p>We now have almost everything we need to process all our data files.
181+
The only thing that’s missing is a library with a rather unpleasant name:</p>
182+
183+
<div class="python highlighter-rouge"><pre class="highlight"><code>import glob
184+
</code></pre>
185+
</div>
186+
187+
<p>The <code class="highlighter-rouge">glob</code> library contains a function, also called <code class="highlighter-rouge">glob</code>,
188+
that finds files and directories whose names match a pattern.
189+
We provide those patterns as strings:
190+
the character <code class="highlighter-rouge">*</code> matches zero or more characters,
191+
while <code class="highlighter-rouge">?</code> matches any one character.
192+
We can use this to get the names of all the CSV files in the current directory:</p>
193+
194+
<div class="python highlighter-rouge"><pre class="highlight"><code>print(glob.glob('data/inflammation*.csv'))
195+
</code></pre>
196+
</div>
197+
198+
<div class="output highlighter-rouge"><pre class="highlight"><code>['data/inflammation-05.csv', 'data/inflammation-11.csv', 'data/inflammation-12.csv', 'data/inflammation-08.csv', 'data/inflammation-03.csv', 'data/inflammation-06.csv', 'data/inflammation-09.csv', 'data/inflammation-07.csv', 'data/inflammation-10.csv', 'data/inflammation-02.csv', 'data/inflammation-04.csv', 'data/inflammation-01.csv']
199+
</code></pre>
200+
</div>
201+
202+
<p>As these examples show,
203+
<code class="highlighter-rouge">glob.glob</code>’s result is a list of file and directory paths in arbitrary order.
204+
This means we can loop over it
205+
to do something with each filename in turn.
206+
In our case,
207+
the “something” we want to do is generate a set of plots for each file in our inflammation dataset.
208+
If we want to start by analyzing just the first three files in alphabetical order, we can use the <code class="highlighter-rouge">sorted</code> built-in function to generate a new sorted list from the <code class="highlighter-rouge">glob.glob</code> output:</p>
209+
210+
<div class="python highlighter-rouge"><pre class="highlight"><code>import numpy
211+
import matplotlib.pyplot
212+
213+
filenames = sorted(glob.glob('data/inflammation*.csv'))
214+
filenames = filenames[0:3]
215+
for f in filenames:
216+
print(f)
217+
218+
data = numpy.loadtxt(fname=f, delimiter=',')
219+
220+
fig = matplotlib.pyplot.figure(figsize=(10.0, 3.0))
221+
222+
axes1 = fig.add_subplot(1, 3, 1)
223+
axes2 = fig.add_subplot(1, 3, 2)
224+
axes3 = fig.add_subplot(1, 3, 3)
225+
226+
axes1.set_ylabel('average')
227+
axes1.plot(numpy.mean(data, axis=0))
228+
229+
axes2.set_ylabel('max')
230+
axes2.plot(numpy.max(data, axis=0))
231+
232+
axes3.set_ylabel('min')
233+
axes3.plot(numpy.min(data, axis=0))
234+
235+
fig.tight_layout()
236+
matplotlib.pyplot.show()
237+
</code></pre>
238+
</div>
239+
240+
<div class="output highlighter-rouge"><pre class="highlight"><code>inflammation-01.csv
241+
</code></pre>
242+
</div>
243+
244+
<p><img src="/swc-releases/2016.06/python-novice-inflammation/fig/03-loop_49_1.png" alt="Analysis of inflammation-01.csv" /></p>
245+
246+
<div class="output highlighter-rouge"><pre class="highlight"><code>inflammation-02.csv
247+
</code></pre>
248+
</div>
249+
250+
<p><img src="/swc-releases/2016.06/python-novice-inflammation/fig/03-loop_49_3.png" alt="Analysis of inflammation-02.csv" /></p>
251+
252+
<div class="output highlighter-rouge"><pre class="highlight"><code>inflammation-03.csv
253+
</code></pre>
254+
</div>
255+
256+
<p><img src="/swc-releases/2016.06/python-novice-inflammation/fig/03-loop_49_5.png" alt="Analysis of inflammation-03.csv" /></p>
257+
258+
<p>Sure enough,
259+
the maxima of the first two data sets show exactly the same ramp as the first,
260+
and their minima show the same staircase structure;
261+
a different situation has been revealed in the third dataset,
262+
where the maxima are a bit less regular, but the minima are consistently zero.</p>
263+
264+
<blockquote class="challenge">
265+
<h2 id="plotting-differences">Plotting Differences</h2>
266+
267+
<p>Plot the difference between the average of the first dataset
268+
and the average of the second dataset,
269+
i.e., the difference between the leftmost plot of the first two figures.</p>
270+
271+
<blockquote class="solution">
272+
<h2 id="solution">Solution</h2>
273+
<div class="python highlighter-rouge"><pre class="highlight"><code>import glob
274+
import numpy
275+
import matplotlib.pyplot
276+
277+
filenames = glob.glob('data/inflammation*.csv')
278+
279+
data0 = numpy.loadtxt(fname=filenames[0], delimiter=',')
280+
data1 = numpy.loadtxt(fname=filenames[1], delimiter=',')
281+
282+
fig = matplotlib.pyplot.figure(figsize=(10.0, 3.0))
283+
284+
matplotlib.pyplot.ylabel('Difference in average')
285+
matplotlib.pyplot.plot(data0.mean(axis=0) - data1.mean(axis=0))
286+
287+
fig.tight_layout()
288+
matplotlib.pyplot.show()
289+
</code></pre>
290+
</div>
291+
</blockquote>
292+
</blockquote>
293+
294+
<blockquote class="challenge">
295+
<h2 id="generate-composite-statistics">Generate Composite Statistics</h2>
296+
297+
<p>Use each of the files once to generate a dataset containing values averaged over all patients:</p>
298+
299+
<div class="python highlighter-rouge"><pre class="highlight"><code>filenames = glob.glob('data/inflammation*.csv')
300+
composite_data = numpy.zeros((60,40))
301+
for f in filenames:
302+
# sum each new file's data into as it's read
303+
# and then divide the composite_data by number of samples
304+
composite_data /= len(filenames)
305+
</code></pre>
306+
</div>
307+
308+
<p>Then use pyplot to generate average, max, and min for all patients.</p>
309+
310+
</blockquote>
311+
312+
<blockquote class="keypoints">
313+
<h2>Key Points</h2>
314+
<ul>
315+
316+
<li><p>Use <code class="highlighter-rouge">glob.glob(pattern)</code> to create a list of files whose names match a pattern.</p>
317+
</li>
318+
319+
<li><p>Use <code class="highlighter-rouge">*</code> in a pattern to match zero or more characters, and <code class="highlighter-rouge">?</code> to match any single character.</p>
320+
</li>
321+
322+
</ul>
323+
</blockquote>
324+
325+
326+
327+
<hr/>
328+
<footer>
329+
<div class="row">
330+
<div class="col-md-6" align="left">
331+
<h4>
332+
Copyright &copy; 2016
333+
<a href="https://software-carpentry.org">Software Carpentry Foundation</a>
334+
335+
</h4>
336+
</div>
337+
<div class="col-md-6" align="right">
338+
<h4>
339+
<a href="/">Source</a>
340+
/
341+
<a href="/blob/gh-pages/CONTRIBUTING.md">Contributing</a>
342+
/
343+
<a href="mailto:[email protected]">Contact</a>
344+
</h4>
345+
</div>
346+
</div>
347+
</footer>
348+
349+
350+
</div>
351+
<script src="/swc-releases/2016.06/python-novice-inflammation/assets/js/jquery.min.js"></script>
352+
<script src="/swc-releases/2016.06/python-novice-inflammation/assets/js/bootstrap.min.js"></script>
353+
<script src="/swc-releases/2016.06/python-novice-inflammation/assets/js/lesson.js"></script>
354+
355+
</body>
356+
</html>

0 commit comments

Comments
 (0)