Fixes to notebooks + Update CHANGELOG

Author: kushalbakshi

CHANGELOG.md

@@ -1,5 +1,9 @@
 # Changelog
 
+## 0.1.4 - 2023-10-11
+
++ Update - Markdown explanations within notebooks `01` and `02`
+
 ## 0.1.3 - 2023-09-14
 
 + Add - GitHub Actions to build Docker image and push to DockerHub

tutorials/01-DataJoint Basics.ipynb

@@ -51,14 +51,17 @@
     "If you visit the [documentation for DataJoint](https://docs.datajoint.io/introduction/Data-pipelines.html), we define a data pipeline as follows:\n",
     "> A data pipeline is a sequence of steps (more generally a directed acyclic graph) with integrated storage at each step. These steps may be thought of as nodes in a graph.\n",
     "\n",
-    ">* Nodes in this graph are represented as database **tables**. Examples of such tables include \"Subject\", \"Session\", \"Implantation\", \"Experimenter\", \"Equipment\", but also \"OptoWaveform\", \"OptoStimParams\", or \"Neuronal spikes\".  \n",
+    ">* Nodes in this graph are represented as database **tables**. Examples of such tables include `Subject`, `Session`, `Implantation`, `Experimenter`, `Equipment`, but also `OptoWaveform`, `OptoStimParams`, or `NeuronalSpikes`.  \n",
     "\n",
     ">* The data pipeline is formed by making these tables interdependent (as the nodes are connected in a network). A **dependency** is a situation where a step of the data pipeline is dependent on a result from a sequentially previous step before it can complete its execution. A dependency graph forms an entire cohesive data pipeline. \n",
     "\n",
-    "1. define these \"things\" as tables in which you can store the information about them\n",
-    "2. define the relationships (in particular the dependencies) between the \"things\"\n",
+    "In order to create a data pipeline, you need to know the \"things\" in your experiments\n",
+    "and the relationship between them. Within the pipeline you will then:\n",
     "\n",
-    "A data pipeline can then serve as a map that describes everything that goes on in your experiment, capturing what is collected, what is processed, and what is analyzed/computed. A well designed data pipeline not only let's you organize your data well, but can bring out logical clarity to your experiment, and may even bring about new insights by making how everything in your experiment relates together obvious.\n",
+    "1. define these \"things\" as tables in which you can store the information about them.\n",
+    "2. define the relationships (in particular the dependencies) between the \"things\".\n",
+    "\n",
+    "The data pipeline can then serve as a map that describes everything that goes on in your experiment, capturing what is collected, what is processed, and what is analyzed/computed. A well designed data pipeline not only let's you organize your data well, but can bring out logical clarity to your experiment, and may even bring about new insights by making how everything in your experiment relates together obvious.\n",
     "\n",
     "Let's go ahead and build together a pipeline from scratch to better understand what a data pipeline is all about."
    ]
@@ -67,7 +70,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "##### Practical examples"
+    "#### Practical examples"
    ]
   },
   {
@@ -131,21 +134,21 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-   "Just by going through the description, we can start to identify **entities** that needs to be stored and represented in our data pipeline:\n",
-   "\n",
-   "* mouse\n",
-   "* experimental session\n",
-   "\n",
-   "For ephys:\n",
-   "\n",
-   ">* neuron\n",
-   ">* spikes\n",
-   "\n",
-   "For calcium imaging:\n",
-   "\n",
-   ">* scan\n",
-   ">* regions of interest\n",
-   ">* trace"
+    "Just by going through the description, we can start to identify **entities** that need to be stored and represented in our data pipeline:\n",
+    "\n",
+    "* mouse\n",
+    "* experimental session\n",
+    "\n",
+    "For ephys:\n",
+    "\n",
+    ">* neuron\n",
+    ">* spikes\n",
+    "\n",
+    "For calcium imaging:\n",
+    "\n",
+    ">* scan\n",
+    ">* regions of interest\n",
+    ">* trace"
    ]
   },
   {
@@ -159,7 +162,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-   "### Schemas and tables"
+    "### Schemas and tables"
    ]
   },
   {
@@ -173,18 +176,21 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-   "In a data pipeline, we represent these **entities** as **tables**. Different *kinds* of entities become distinct tables, and each table row is a single example (instance) of the entity's category. \n",
-   "\n",
-   "For example, if we have a `Mouse` table, each row in the mouse table represents a single mouse. \n",
-   "\n",
-   "It is essential to think about what information will **uniquely identify** each entry. \n",
-   "\n",
-   "In this case, the information that uniquely identifies the `Mouse` table is their **mouse IDs** - a unique ID number assigned to each animal in the lab. This attribute is named the **primary key** of the table.\n",
-   "\n",
-   "| Mouse_ID (*Primary key attribute*)|\n",
-   "|:--------: |                         \n",
-   "| 11234     |\n",
-   "| 11432     |"
+    "In a data pipeline, we represent these **entities** as **tables**. Different *kinds* of entities become distinct tables, and each table row is a single example (instance) of the entity's category. \n",
+    "\n",
+    "For example, if we have a `Mouse` table, each row in the mouse table represents a single mouse. \n",
+    "\n",
+    "It is essential to think about what information will **uniquely identify** each entry. \n",
+    "\n",
+    "In this case, the information that uniquely identifies the `Mouse` table is their\n",
+    "**mouse ID** - a unique ID number assigned to each animal in the lab. This attribute is\n",
+    "named the **primary key** of the table. By convention, table attributes are lower case\n",
+    "and do not contain spaces.\n",
+    "\n",
+    "| `mouse_id*` (*Primary key attribute*)|\n",
+    "|:--------: |                         \n",
+    "| 11234     |\n",
+    "| 11432     |"
    ]
   },
   {
@@ -197,7 +203,7 @@
     "\n",
     "Such an attribute is called the **primary key** of the table: the subset of table attributes uniquely identifying each entity in the table. The **secondary attribute** refers to any field in a table, not in the primary key.\n",
     "\n",
-    "| Mouse_ID (*Primary key attribute*) \n",
+    "| `mouse_id*` (*Primary key attribute*) \n",
     "|:--------:|  \n",
     "| 11234      (*Secondary attribute*)\n",
     "| 11432      (*Secondary attribute*)"
@@ -207,7 +213,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-   "Once we have successfully identified the table's primary key, we can now think about what other columns, or **non-primary key attributes** - additional information **about each entry in the table that need to be stored as well**."
+    "Once we have successfully identified the table's primary key, we can now think about what other columns, or **non-primary key attributes** - additional information **about each entry in the table that need to be stored as well**."
    ]
   },
   {
@@ -221,7 +227,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "| Mouse_ID | DOB        | sex |\n",
+    "| `mouse_id*` | `dob`        | `sex` |\n",
     "|:--------:|------------|--------|\n",
     "| 11234    | 2017-11-17 | M      |\n",
     "| 11432    | 2018-03-04 | F      |"
@@ -231,7 +237,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-   "Now that we have an idea of how to represent information about the mouse, let's create the table using **DataJoint**!"
+    "Now that we have an idea of how to represent information about the mouse, let's create the table using **DataJoint**!"
    ]
   },
   {
@@ -245,7 +251,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-   "##### Schema"
+    "##### Schema"
    ]
   },
   {
@@ -283,14 +289,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-   "##### Table"
+    "##### Table"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-   "In DataJoint, you define each table as a `class`, and provide the table definition (e.g., attribute definitions) as the `definition` static string property. The class will inherit from the `dj.Manual` class provided by DataJoint (more on this later)."
+    "In DataJoint, you define each table as a `class`, and provide the table definition (e.g. attribute definitions) as the `definition` static string property. The class will inherit from the `dj.Manual` class provided by DataJoint (more on this later)."
    ]
   },
   {
@@ -330,7 +336,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-   "### Basic relational operators"
+    "### Basic relational operators"
    ]
   },
   {
@@ -477,7 +483,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-   "##### Data integrity"
+    "##### Data integrity"
    ]
   },
   {
@@ -563,31 +569,29 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-   "As with `mouse`, we should consider **what information (i.e., attributes) is needed to identify an `experimental session`** uniquely. Here is the relevant section of the project description:\n",
-   "\n",
-   "> * As a hard working neuroscientist, you perform experiments every day, sometimes working with **more than one mouse in a day**! However, on an any given day, **a mouse undergoes at most one recording session**.\n",
-   "> * For each experimental session, you would like to record **what mouse you worked with** and **when you performed the experiment**. You would also like to keep track of other helpful information such as the **experimental setup** you worked on."
+    "As with `mouse`, we should consider **what information (i.e. attributes) is needed to identify an experimental `session`** uniquely. Here is the relevant section of the project description:\n",
+    "\n",
+    "> * As a hard working neuroscientist, you perform experiments every day, sometimes working with **more than one mouse in a day**! However, on an any given day, **a mouse undergoes at most one recording session**.\n",
+    "> * For each experimental session, you would like to record **what mouse you worked with** and **when you performed the experiment**. You would also like to keep track of other helpful information such as the **experimental setup** you worked on."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-   "Based on the above, it seems that you need to know these two data to uniquely identify a single experimental session:\n",
-   "\n",
-   "* the date of the session\n",
-   "* the mouse you recorded from in that session\n",
-   "\n",
-   "to uniquely identify a single experimental session."
+    "Based on the above, it seems that you need to know the following data to uniquely identify a single experimental session:\n",
+    "\n",
+    "* the date of the session\n",
+    "* the mouse you recorded from in that session"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-   "Note that, to uniquely identify an experimental session (or simply a `Session`), we need to know the mouse that the session was about. In other words, a session cannot existing without a corresponding mouse! \n",
-   "\n",
-   "With **mouse** already represented as a table in our pipeline, we say that the session **depends on** the mouse! We could graphically represent this in an **entity relationship diagram (ERD)** by drawing the line between two tables, with the one below (**session**) depending on the one above (**mouse**)."
+    "Note that, to uniquely identify an experimental session (or simply a `Session`), we need to know the mouse that the session was about. In other words, a session cannot exist without a corresponding mouse! \n",
+    "\n",
+    "With **mouse** already represented as a table in our pipeline, we say that the session **depends on** the mouse! We could graphically represent this in an **entity relationship diagram (ERD)** by drawing the line between two tables, with the one below (**session**) depending on the one above (**mouse**)."
    ]
   },
   {
@@ -778,13 +782,17 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We will introduce significant types of queries used in DataJoint:\n",
-    "* 1. Restriction (`&`) and negative restriction (`-`): filter the data with certain conditions\n",
-    "* 2. Join (`*`): bring fields from different tables together\n",
-    "* 3. Projection (`.proj()`): focus on a subset of attributes\n",
-    "* 4. Fetch (`.fetch()`): pull the data from the database\n",
-    "* 5. Deletion (`.delete()`): delete entries and their dependencies\n",
-    "* 6. Drop (`.drop()`): drop the table from the schema"
+    "We will introduce the major types of queries used in DataJoint:\n",
+    "1. Restriction (`&`) and negative restriction (`-`): filter the data with certain conditions\n",
+    "2. Join (`*`): bring fields from different tables together\n",
+    "3. Projection (`.proj()`): focus on a subset of attributes\n",
+    "\n",
+    "Following the query operations, you might work with one or more of the following\n",
+    "data manipulation operations supported by DataJoint:\n",
+    " \n",
+    "1. Fetch (`.fetch()`): pull the data from the database\n",
+    "2. Deletion (`.delete()`): delete entries and their dependencies\n",
+    "3. Drop (`.drop()`): drop the table from the schema"
    ]
   },
   {
@@ -805,7 +813,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "##### Exact match"
+    "#### Exact match"
    ]
   },
   {
@@ -876,7 +884,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Inequality"
+    "#### Inequality"
    ]
   },
   {
@@ -1010,7 +1018,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Restriction one table with another"
+    "#### Restrict one table with another"
    ]
   },
   {
@@ -1033,7 +1041,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Combining restrictions"
+    "#### Combine restrictions"
    ]
   },
   {
@@ -1079,7 +1087,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Negative restriction - with the `-` operator"
+    "#### Negative restriction: with the `-` operator"
    ]
   },
   {
@@ -1134,10 +1142,10 @@
    "source": [
     "Behavior of join:\n",
     "\n",
-    "1. match the common field(s) of the primary keys in the two tables\n",
-    "2. do a combination of the non-matched part of the primary key\n",
-    "3. listing out the secondary attributes for each combination\n",
-    "4. if two tables have secondary attributes that share a same name, it will throw an error. To join, we need to rename that attribute for at least one of the tables."
+    "1. Match the common field(s) of the primary keys in the two tables.\n",
+    "2. Do a combination of the non-matched part of the primary key.\n",
+    "3. Listing out the secondary attributes for each combination.\n",
+    "4. If two tables have secondary attributes that share a same name, it will throw an error. To join, we need to rename that attribute for at least one of the tables."
    ]
   },
   {