|
45 | 45 | ] |
46 | 46 | }, |
47 | 47 | { |
| 48 | + "attachments": {}, |
48 | 49 | "cell_type": "markdown", |
49 | 50 | "metadata": {}, |
50 | 51 | "source": [ |
51 | 52 | "If you visit the [documentation for DataJoint](https://docs.datajoint.io/introduction/Data-pipelines.html), we define a data pipeline as follows:\n", |
52 | 53 | "> A data pipeline is a sequence of steps (more generally a directed acyclic graph) with integrated storage at each step. These steps may be thought of as nodes in a graph.\n", |
53 | 54 | "\n", |
54 | | - "While this is an accurate description, it may not be the most intuitive definition. Put succinctly, a data pipeline is a listing or a \"map\" of various \"things\" that you work with in a project, with line connecting things to each other to indicate their dependecies. The \"things\" in a data pipeline tends to be the *nouns* you find when describing a project. The \"things\" may include anything from mouse, experimenter, equipment, to experiment session, trial, two-photon scans, electric activities, to receptive fields, neuronal spikes, to figures for a publication! A data pipeline gives you a framework to:\n", |
| 55 | + "While this is an accurate description, it may not be the most intuitive definition. Put succinctly, a data pipeline is a listing or a \"map\" of various \"things\" that you work with in a project, with line connecting things to each other to indicate their dependencies. The \"things\" in a data pipeline tends to be the *nouns* you find when describing a project. The \"things\" may include anything from mouse, experimenter, equipment, to experiment session, trial, two-photon scans, electric activities, to receptive fields, neuronal spikes, to figures for a publication! A data pipeline gives you a framework to:\n", |
55 | 56 | "\n", |
56 | 57 | "1. define these \"things\" as tables in which you can store the information about them\n", |
57 | 58 | "2. define the relationships (in particular the dependencies) between the \"things\"\n", |
|
76 | 77 | ] |
77 | 78 | }, |
78 | 79 | { |
| 80 | + "attachments": {}, |
79 | 81 | "cell_type": "markdown", |
80 | 82 | "metadata": {}, |
81 | 83 | "source": [ |
|
86 | 88 | "> * In a session of electrophysiology\n", |
87 | 89 | ">> * you record electrical activity from a single neuron. You use recording equipment that produces separate data files for each neuron you recorded.\n", |
88 | 90 | ">> * Neuron's activities are recorded as raw traces. Neuron's spikes needs to be detected for further analysis to be performed.\n", |
89 | | - "> * In a sesssion of calcium imaging\n", |
| 91 | + "> * In a session of calcium imaging\n", |
90 | 92 | ">> * you scan a brain region containing a number of neurons. You use recording equipment that produces separate data files for each scan you performed.\n", |
91 | 93 | ">> * you would like to segment the frames and get the regions of interest (ROIs), and save a mask for each ROI\n", |
92 | 94 | ">> * finally you would like to extract the trace from each segmented ROI" |
|
107 | 109 | ] |
108 | 110 | }, |
109 | 111 | { |
| 112 | + "attachments": {}, |
110 | 113 | "cell_type": "markdown", |
111 | 114 | "metadata": {}, |
112 | 115 | "source": [ |
|
117 | 120 | "> * In a session of electrophysiology\n", |
118 | 121 | ">> * you record electrical activity from a **single neuron**. You use recording equipment that produces separate data files for each neuron you recorded.\n", |
119 | 122 | ">> * Neuron's activities are recorded as raw traces. **Neuron's spikes** needs to be detected for further analysis to be performed.\n", |
120 | | - "> * In a sesssion of calcium imaging\n", |
| 123 | + "> * In a session of calcium imaging\n", |
121 | 124 | ">> * you scan a brain region containing a number of neurons. You use recording equipment that produces separate data files for each **scan** you performed.\n", |
122 | 125 | ">> * you would like to segment the frames and get the **regions of interest (ROIs)**, and save a mask for each ROI\n", |
123 | 126 | ">> * finally you would like to extract the **trace** from each segmented ROI" |
|
1053 | 1056 | ] |
1054 | 1057 | }, |
1055 | 1058 | { |
| 1059 | + "attachments": {}, |
1056 | 1060 | "cell_type": "markdown", |
1057 | 1061 | "metadata": {}, |
1058 | 1062 | "source": [ |
1059 | 1063 | "Note that, to uniquely identify an experimental session (or simply a **session**), we need to know the mouse that the session was about. In other words, a session cannot existing without a corresponding mouse! \n", |
1060 | 1064 | "\n", |
1061 | | - "With **mouse** already represented as a table in our pipeline, we say that the session **depends on** the mouse! We could graphically represent this in an **entity relationship diagram (ERD)** by drawing the line between two tables, with the one below (**session**) dependeing on the one above (**mouse**)." |
| 1065 | + "With **mouse** already represented as a table in our pipeline, we say that the session **depends on** the mouse! We could graphically represent this in an **entity relationship diagram (ERD)** by drawing the line between two tables, with the one below (**session**) depending on the one above (**mouse**)." |
1062 | 1066 | ] |
1063 | 1067 | }, |
1064 | 1068 | { |
|
1114 | 1118 | } |
1115 | 1119 | ], |
1116 | 1120 | "source": [ |
1117 | | - "dj.ERD(schema)" |
| 1121 | + "dj.Diagram(schema)" |
1118 | 1122 | ] |
1119 | 1123 | }, |
1120 | 1124 | { |
|
4006 | 4010 | ] |
4007 | 4011 | }, |
4008 | 4012 | { |
| 4013 | + "attachments": {}, |
4009 | 4014 | "cell_type": "markdown", |
4010 | 4015 | "metadata": {}, |
4011 | 4016 | "source": [ |
4012 | 4017 | "### Rename attribute with proj()\n", |
4013 | | - "Say we want to rename the exisiting attribute `dob` of the `Mouse` table to `date_of_birth`, this can be done using `.proj()`" |
| 4018 | + "Say we want to rename the existing attribute `dob` of the `Mouse` table to `date_of_birth`, this can be done using `.proj()`" |
4014 | 4019 | ] |
4015 | 4020 | }, |
4016 | 4021 | { |
|
4238 | 4243 | ] |
4239 | 4244 | }, |
4240 | 4245 | { |
| 4246 | + "attachments": {}, |
4241 | 4247 | "cell_type": "markdown", |
4242 | 4248 | "metadata": {}, |
4243 | 4249 | "source": [ |
4244 | | - "Note: as you can see, the projection results keep the primary attributes from the `Mouse * Session` joinning operation, while removing all other non-primary attributes. To Keep all other attributes, you can use the `...` syntax" |
| 4250 | + "Note: as you can see, the projection results keep the primary attributes from the `Mouse * Session` joining operation, while removing all other non-primary attributes. To Keep all other attributes, you can use the `...` syntax" |
4245 | 4251 | ] |
4246 | 4252 | }, |
4247 | 4253 | { |
|
0 commit comments