Skip to content

Commit 3cfea31

Browse files
MilagrosMarinMilagrosMarin
committed
02Tut: section added,code moved from mouse_session
1 parent c164981 commit 3cfea31

File tree

1 file changed

+203
-79
lines changed

1 file changed

+203
-79
lines changed

‎notebooks/02-Calcium Imaging Imported Tables.ipynb‎

Lines changed: 203 additions & 79 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@
3636
},
3737
{
3838
"cell_type": "code",
39-
"execution_count": 1,
39+
"execution_count": 2,
4040
"metadata": {},
4141
"outputs": [],
4242
"source": [
@@ -52,7 +52,7 @@
5252
},
5353
{
5454
"cell_type": "code",
55-
"execution_count": 2,
55+
"execution_count": 3,
5656
"metadata": {},
5757
"outputs": [],
5858
"source": [
@@ -65,36 +65,185 @@
6565
"cell_type": "markdown",
6666
"metadata": {},
6767
"source": [
68-
"## Schema, Mouse & Session"
68+
"## Calcium imaging dataset"
6969
]
7070
},
7171
{
7272
"cell_type": "markdown",
7373
"metadata": {},
7474
"source": [
75-
"We will continue working with the tables that we defined in the previous session. To do so, we will need the classes for each table: `Mouse` and `Session`. \n",
75+
"In the `data` folder in this `DataJoint-Tutorials`, you can find a small dataset of three different cases of calcium imaging scans:\n",
7676
"\n",
77-
"We can either redefine it here, but for your convenience, we have included the schema and table class definitions in a package called `tutorial_pipeline.mouse_session`, from which you can import the classes as well as the schema object. \n",
77+
"- `/workspaces/datajoint-tutorials/data/example_scan_02.tif`\n",
78+
"- `/workspaces/datajoint-tutorials/data/example_scan_03.tif`\n",
79+
"- `/workspaces/datajoint-tutorials/data/example_scan_01.tif`\n",
7880
"\n",
79-
"We will use the schema object again to define more tables."
81+
"As you might know, calcium imaging scans (raw data) are stored as *.tif* files. \n",
82+
"\n",
83+
"*NOTE: For this tutorial there is no need to deeper explore this small dataset. Nevertheless, if you are curious about visualizing these example scans, we recommend you to open the TIFF with [ImageJ](https://imagej.nih.gov/ij/download.html).*"
84+
]
85+
},
86+
{
87+
"cell_type": "markdown",
88+
"metadata": {},
89+
"source": [
90+
"## Pipeline design: Schema, Mouse & Session"
91+
]
92+
},
93+
{
94+
"cell_type": "markdown",
95+
"metadata": {},
96+
"source": [
97+
"The DataJoint pipeline commonly starts with a `schema` and the following classes for each table: `Mouse` and `Session`. Let's quickly create this pipeline's first steps as we learned it in the previous session:"
8098
]
8199
},
82100
{
83101
"cell_type": "code",
84-
"execution_count": 3,
102+
"execution_count": 4,
85103
"metadata": {},
86104
"outputs": [
87105
{
88-
"name": "stderr",
89-
"output_type": "stream",
90-
"text": [
91-
"[2023-07-25 20:54:56,914][INFO]: Connecting [email protected]:3306\n",
92-
"[2023-07-25 20:54:56,935][INFO]: Connected [email protected]:3306\n"
106+
"ename": "NameError",
107+
"evalue": "name 'Mouse' is not defined",
108+
"output_type": "error",
109+
"traceback": [
110+
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
111+
"\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
112+
"Cell \u001b[0;32mIn[4], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m Mouse\u001b[39m.\u001b[39mdelete()\n",
113+
"\u001b[0;31mNameError\u001b[0m: name 'Mouse' is not defined"
93114
]
94115
}
95116
],
96117
"source": [
97-
"from tutorial_pipeline.mouse_session import schema, Mouse, Session"
118+
"Mouse.delete()"
119+
]
120+
},
121+
{
122+
"cell_type": "code",
123+
"execution_count": 22,
124+
"metadata": {},
125+
"outputs": [],
126+
"source": [
127+
"schema = dj.schema('tutorial')"
128+
]
129+
},
130+
{
131+
"cell_type": "code",
132+
"execution_count": 23,
133+
"metadata": {},
134+
"outputs": [],
135+
"source": [
136+
"@schema\n",
137+
"class Mouse(dj.Manual):\n",
138+
" definition = \"\"\"\n",
139+
" # Experimental animals\n",
140+
" mouse_id : int # Unique animal ID\n",
141+
" ---\n",
142+
" dob=null : date # date of birth\n",
143+
" sex=\"unknown\" : enum('M','F','unknown') # sex\n",
144+
" \"\"\"\n"
145+
]
146+
},
147+
{
148+
"cell_type": "markdown",
149+
"metadata": {},
150+
"source": [
151+
"Insert the following example data into the table:"
152+
]
153+
},
154+
{
155+
"cell_type": "code",
156+
"execution_count": 24,
157+
"metadata": {},
158+
"outputs": [],
159+
"source": [
160+
"mouse_data = [\n",
161+
" {'dob': \"2017-03-01\", 'mouse_id': 0, 'sex': 'M'},\n",
162+
" {'dob': \"2016-11-19\", 'mouse_id': 1, 'sex': 'M'},\n",
163+
" {'dob': \"2016-11-20\", 'mouse_id': 2, 'sex': 'unknown'},\n",
164+
" {'dob': \"2016-12-25\", 'mouse_id': 5, 'sex': 'F'},\n",
165+
" {'dob': \"2017-01-01\", 'mouse_id': 10, 'sex': 'F'},\n",
166+
" {'dob': \"2017-01-03\", 'mouse_id': 11, 'sex': 'F'},\n",
167+
" {'dob': \"2017-05-12\", 'mouse_id': 100, 'sex': 'F'}\n",
168+
"]"
169+
]
170+
},
171+
{
172+
"cell_type": "code",
173+
"execution_count": 25,
174+
"metadata": {},
175+
"outputs": [],
176+
"source": [
177+
"Mouse.insert(mouse_data, skip_duplicates=True)"
178+
]
179+
},
180+
{
181+
"cell_type": "code",
182+
"execution_count": 26,
183+
"metadata": {},
184+
"outputs": [],
185+
"source": [
186+
"@schema\n",
187+
"class Session(dj.Manual):\n",
188+
" definition = \"\"\"\n",
189+
" # Experiment session\n",
190+
" -> Mouse\n",
191+
" session_date : date # date\n",
192+
" ---\n",
193+
" experiment_setup : int # experiment setup ID\n",
194+
" experimenter : varchar(100) # experimenter name\n",
195+
" data_path='' : varchar(255) #\n",
196+
" \"\"\"\n"
197+
]
198+
},
199+
{
200+
"cell_type": "markdown",
201+
"metadata": {},
202+
"source": [
203+
"Insert the following example data into the table:"
204+
]
205+
},
206+
{
207+
"cell_type": "code",
208+
"execution_count": 27,
209+
"metadata": {},
210+
"outputs": [],
211+
"source": [
212+
"session_data = [\n",
213+
" {'experiment_setup': 0,\n",
214+
" 'experimenter': 'Edgar Y. Walker',\n",
215+
" 'mouse_id': 0,\n",
216+
" 'session_date': \"2017-05-15\",\n",
217+
" 'data_path': data_dir.as_posix()\n",
218+
" },\n",
219+
" {'experiment_setup': 0,\n",
220+
" 'experimenter': 'Edgar Y. Walker',\n",
221+
" 'mouse_id': 0,\n",
222+
" 'session_date': \"2017-05-19\",\n",
223+
" 'data_path': data_dir.as_posix()\n",
224+
" },\n",
225+
" {'experiment_setup': 1,\n",
226+
" 'experimenter': 'Fabian Sinz',\n",
227+
" 'mouse_id': 5,\n",
228+
" 'session_date': \"2017-01-05\",\n",
229+
" 'data_path': data_dir.as_posix()\n",
230+
" },\n",
231+
" {'experiment_setup': 100,\n",
232+
" 'experimenter': 'Jacob Reimer',\n",
233+
" 'mouse_id': 100,\n",
234+
" 'session_date': \"2017-05-25\",\n",
235+
" 'data_path': data_dir.as_posix()\n",
236+
" }\n",
237+
"]"
238+
]
239+
},
240+
{
241+
"cell_type": "code",
242+
"execution_count": 28,
243+
"metadata": {},
244+
"outputs": [],
245+
"source": [
246+
"Session.insert(session_data, skip_duplicates=True)"
98247
]
99248
},
100249
{
@@ -106,7 +255,7 @@
106255
},
107256
{
108257
"cell_type": "code",
109-
"execution_count": 4,
258+
"execution_count": 29,
110259
"metadata": {},
111260
"outputs": [
112261
{
@@ -189,12 +338,22 @@
189338
"<td>2017-01-01</td>\n",
190339
"<td>F</td></tr><tr><td>11</td>\n",
191340
"<td>2017-01-03</td>\n",
192-
"<td>F</td></tr><tr><td>100</td>\n",
341+
"<td>F</td></tr><tr><td>12</td>\n",
342+
"<td>2017-03-21</td>\n",
343+
"<td>F</td></tr><tr><td>18</td>\n",
344+
"<td>2017-05-01</td>\n",
345+
"<td>F</td></tr><tr><td>19</td>\n",
346+
"<td>2018-07-21</td>\n",
347+
"<td>M</td></tr><tr><td>22</td>\n",
348+
"<td>2019-12-15</td>\n",
349+
"<td>F</td></tr><tr><td>34</td>\n",
350+
"<td>2018-09-22</td>\n",
351+
"<td>M</td></tr><tr><td>100</td>\n",
193352
"<td>2017-05-12</td>\n",
194353
"<td>F</td> </tr> </tbody>\n",
195354
" </table>\n",
196355
" \n",
197-
" <p>Total: 7</p></div>\n",
356+
" <p>Total: 12</p></div>\n",
198357
" "
199358
],
200359
"text/plain": [
@@ -206,11 +365,16 @@
206365
"5 2016-12-25 F \n",
207366
"10 2017-01-01 F \n",
208367
"11 2017-01-03 F \n",
368+
"12 2017-03-21 F \n",
369+
"18 2017-05-01 F \n",
370+
"19 2018-07-21 M \n",
371+
"22 2019-12-15 F \n",
372+
"34 2018-09-22 M \n",
209373
"100 2017-05-12 F \n",
210-
" (Total: 7)"
374+
" (Total: 12)"
211375
]
212376
},
213-
"execution_count": 4,
377+
"execution_count": 29,
214378
"metadata": {},
215379
"output_type": "execute_result"
216380
}
@@ -221,7 +385,7 @@
221385
},
222386
{
223387
"cell_type": "code",
224-
"execution_count": 6,
388+
"execution_count": 30,
225389
"metadata": {},
226390
"outputs": [
227391
{
@@ -302,35 +466,45 @@
302466
"<td>2017-05-15</td>\n",
303467
"<td>0</td>\n",
304468
"<td>Edgar Y. Walker</td>\n",
305-
"<td>/workspaces/datajoint-tutorials/data</td></tr><tr><td>0</td>\n",
469+
"<td></td></tr><tr><td>0</td>\n",
306470
"<td>2017-05-19</td>\n",
307471
"<td>0</td>\n",
308472
"<td>Edgar Y. Walker</td>\n",
309-
"<td>/workspaces/datajoint-tutorials/data</td></tr><tr><td>5</td>\n",
473+
"<td>/workspaces/datajoint-tutorials/data</td></tr><tr><td>0</td>\n",
474+
"<td>2018-01-15</td>\n",
475+
"<td>100</td>\n",
476+
"<td>Jacob Reimer</td>\n",
477+
"<td></td></tr><tr><td>5</td>\n",
310478
"<td>2017-01-05</td>\n",
311479
"<td>1</td>\n",
312480
"<td>Fabian Sinz</td>\n",
313-
"<td>/workspaces/datajoint-tutorials/data</td></tr><tr><td>100</td>\n",
481+
"<td>/workspaces/datajoint-tutorials/data</td></tr><tr><td>18</td>\n",
482+
"<td>2018-01-15</td>\n",
483+
"<td>101</td>\n",
484+
"<td>Jacob Reimer</td>\n",
485+
"<td></td></tr><tr><td>100</td>\n",
314486
"<td>2017-05-25</td>\n",
315487
"<td>100</td>\n",
316488
"<td>Jacob Reimer</td>\n",
317489
"<td>/workspaces/datajoint-tutorials/data</td> </tr> </tbody>\n",
318490
" </table>\n",
319491
" \n",
320-
" <p>Total: 4</p></div>\n",
492+
" <p>Total: 6</p></div>\n",
321493
" "
322494
],
323495
"text/plain": [
324496
"*mouse_id *session_date experiment_set experimenter data_path \n",
325497
"+----------+ +------------+ +------------+ +------------+ +------------+\n",
326-
"0 2017-05-15 0 Edgar Y. Walke /workspaces/da\n",
498+
"0 2017-05-15 0 Edgar Y. Walke \n",
327499
"0 2017-05-19 0 Edgar Y. Walke /workspaces/da\n",
500+
"0 2018-01-15 100 Jacob Reimer \n",
328501
"5 2017-01-05 1 Fabian Sinz /workspaces/da\n",
502+
"18 2018-01-15 101 Jacob Reimer \n",
329503
"100 2017-05-25 100 Jacob Reimer /workspaces/da\n",
330-
" (Total: 4)"
504+
" (Total: 6)"
331505
]
332506
},
333-
"execution_count": 6,
507+
"execution_count": 30,
334508
"metadata": {},
335509
"output_type": "execute_result"
336510
}
@@ -465,62 +639,12 @@
465639
"The underline `____` indicates **additional primary key attribute(s)** apart from the ones inherited from its parents."
466640
]
467641
},
468-
{
469-
"cell_type": "markdown",
470-
"metadata": {},
471-
"source": [
472-
"## Calcium imaging datasets"
473-
]
474-
},
475-
{
476-
"cell_type": "markdown",
477-
"metadata": {},
478-
"source": [
479-
"In the `data` folder in this `DataJoint-Tutorials`, you can find three calcium imaging datasets. Raw data are stored as *.tif* files.\n",
480-
"\n",
481-
"Let's list these three files:"
482-
]
483-
},
484-
{
485-
"cell_type": "code",
486-
"execution_count": 9,
487-
"metadata": {},
488-
"outputs": [
489-
{
490-
"data": {
491-
"text/plain": [
492-
"PosixPath('/workspaces/datajoint-tutorials/data')"
493-
]
494-
},
495-
"execution_count": 9,
496-
"metadata": {},
497-
"output_type": "execute_result"
498-
}
499-
],
500-
"source": [
501-
"from tutorial_pipeline import data_dir\n",
502-
"data_dir"
503-
]
504-
},
505642
{
506643
"cell_type": "code",
507-
"execution_count": 10,
644+
"execution_count": null,
508645
"metadata": {},
509-
"outputs": [
510-
{
511-
"name": "stdout",
512-
"output_type": "stream",
513-
"text": [
514-
"/workspaces/datajoint-tutorials/data/example_scan_02.tif\n",
515-
"/workspaces/datajoint-tutorials/data/example_scan_03.tif\n",
516-
"/workspaces/datajoint-tutorials/data/example_scan_01.tif\n"
517-
]
518-
}
519-
],
520-
"source": [
521-
"for f in data_dir.glob('*.tif'):\n",
522-
" print(f)"
523-
]
646+
"outputs": [],
647+
"source": []
524648
},
525649
{
526650
"cell_type": "code",

0 commit comments

Comments
 (0)