Merge branch 'dimitri-yatsenko:main' into main

dimitri-yatsenko · web-flow · commit f65bc4f77e95 · 2025-12-16T12:20:34.000-06:00
diff --git a/book/30-design/010-schema.ipynb b/book/30-design/010-schema.ipynb
@@ -3,7 +3,7 @@
   {
    "cell_type": "markdown",
    "metadata": {},
-   "source": "---\ntitle: Create Schemas\nauthors:\n  - name: Dimitri Yatsenko\n---\n\n# What is a schema?\n\nThe term schema has two related meanings in the context of databases:\n\n## 1. Schema as a Data Blueprint\nA **schema** is a formal specification of the structure of data and the rules governing its integrity.\nIt serves as a blueprint that defines how data is organized, stored, and accessed within a database.\nThis ensures that the database reflects the rules and requirements of the underlying business or research project it supports.\n\nIn structured data models, such as the relational model, a schema provides a robust framework for defining:\n* The structure of tables (relations) and their attributes (columns).\n* Rules and constraints that ensure data consistency, accuracy, and reliability.\n* Relationships between tables, such as primary keys (unique identifiers for records) and foreign keys (references to related records in other tables).\n\n### Aims of Good Schema Design\n* **Data Integrity**: Ensures consistency and prevents anomalies.\n* **Query Efficiency**: Facilitates fast and accurate data retrieval, supports complex queries, and optimizes database performance.\n* **Scalability**: Allows the database to grow and adapt as data volumes increase.\n\n### Key Elements of Schema Design\n* **Tables and Attributes**: Each table is defined with specific attributes (columns), each assigned a data type.\n* **Primary Keys**: Uniquely identify each record in a table.\n* **Foreign Keys**: Establish relationships between entities in tables.\n* **Indexes**: Support efficient queries.\n\nThrough careful schema design, database architects create systems that are both efficient and flexible, meeting the current and future needs of an organization. The schema acts as a living document that guides the structure, operations, and integrity of the database.\n\n## 2. Schema as a Database Module\n\nIn complex database designs, the term \"schema\" is also used to describe a distinct module of a larger database with its own namespace that groups related tables together. \nThis modular approach:\n* Separates tables into logical groups for better organization.\n* Avoids naming conflicts in large databases with multiple schemas."
+   "source": "---\ntitle: Schemas\nauthors:\n  - name: Dimitri Yatsenko\n---\n\n# What is a schema?\n\nThe term schema has two related meanings in the context of databases:\n\n## 1. Schema as a Data Blueprint\nA **schema** is a formal specification of the structure of data and the rules governing its integrity.\nIt serves as a blueprint that defines how data is organized, stored, and accessed within a database.\nThis ensures that the database reflects the rules and requirements of the underlying business or research project it supports.\n\nIn structured data models, such as the relational model, a schema provides a robust framework for defining:\n* The structure of tables (relations) and their attributes (columns).\n* Rules and constraints that ensure data consistency, accuracy, and reliability.\n* Relationships between tables, such as primary keys (unique identifiers for records) and foreign keys (references to related records in other tables).\n\n### Aims of Good Schema Design\n* **Data Integrity**: Ensures consistency and prevents anomalies.\n* **Query Efficiency**: Facilitates fast and accurate data retrieval, supports complex queries, and optimizes database performance.\n* **Scalability**: Allows the database to grow and adapt as data volumes increase.\n\n### Key Elements of Schema Design\n* **Tables and Attributes**: Each table is defined with specific attributes (columns), each assigned a data type.\n* **Primary Keys**: Uniquely identify each record in a table.\n* **Foreign Keys**: Establish relationships between entities in tables.\n* **Indexes**: Support efficient queries.\n\nThrough careful schema design, database architects create systems that are both efficient and flexible, meeting the current and future needs of an organization. The schema acts as a living document that guides the structure, operations, and integrity of the database.\n\n## 2. Schema as a Database Module\n\nIn complex database designs, the term \"schema\" is also used to describe a distinct module of a larger database with its own namespace that groups related tables together. \nThis modular approach:\n* Separates tables into logical groups for better organization.\n* Avoids naming conflicts in large databases with multiple schemas."
   },
   {
    "cell_type": "markdown",
diff --git a/book/30-design/015-table.ipynb b/book/30-design/015-table.ipynb
@@ -8,11 +8,7 @@
   {
    "cell_type": "markdown",
    "metadata": {},
-   "source": [
-    "# Prerequisites: Schema Declaration\n",
-    "\n",
-    "Before creating tables, we must declare a schema object that provides a namespace for our tables (see [Create Schemas](010-schema.ipynb)):"
-   ]
+   "source": "# Prerequisites: Schema Declaration\n\nBefore creating tables, we must declare a schema object that provides a namespace for our tables (see [Schemas](010-schema.ipynb)):"
   },
   {
    "cell_type": "code",
@@ -328,19 +324,7 @@
   {
    "cell_type": "markdown",
    "metadata": {},
-   "source": [
-    "```{warning}\n",
-    "**This action is irreversible!**\n",
-    "\n",
-    "- Removes table structure and all data\n",
-    "- Cascades to dependent tables if they exist\n",
-    "- Requires kernel restart after dropping\n",
-    "\n",
-    "Always back up critical data before dropping tables.\n",
-    "```\n",
-    "\n",
-    "For information on dropping entire schemas, see [Create Schemas](010-schema.ipynb)."
-   ]
+   "source": "```{warning}\n**This action is irreversible!**\n\n- Removes table structure and all data\n- Cascades to dependent tables if they exist\n- Requires kernel restart after dropping\n\nAlways back up critical data before dropping tables.\n```\n\nFor information on dropping entire schemas, see [Schemas](010-schema.ipynb)."
   },
   {
    "cell_type": "code",
diff --git a/book/30-design/090-pipeline-project.md b/book/30-design/090-pipeline-project.md
@@ -314,7 +314,7 @@ A production DataJoint pipeline project:
 For simple scripts and learning, see the examples throughout this book. Use this full project structure when you're ready for production deployment.
 
 :::{seealso}
-- [Create Schemas](010-schema.ipynb) — Declaring schemas and tables
+- [Schemas](010-schema.ipynb) — Declaring schemas and tables
 - [Orchestration](../40-operations/060-orchestration.ipynb) — Running pipelines at scale
 - [DataJoint Specs](../95-reference/SPECS_2_0.md) — Complete specification reference
 :::

Original file line number	Diff line number	Diff line change
`@@ -3,7 +3,7 @@`
`3`	`3`	`{`
`4`	`4`	`"cell_type": "markdown",`
`5`	`5`	`"metadata": {},`
`6`		- "source": "---\ntitle: Create Schemas\nauthors:\n - name: Dimitri Yatsenko\n---\n\n# What is a schema?\n\nThe term schema has two related meanings in the context of databases:\n\n## 1. Schema as a Data Blueprint\nA schema is a formal specification of the structure of data and the rules governing its integrity.\nIt serves as a blueprint that defines how data is organized, stored, and accessed within a database.\nThis ensures that the database reflects the rules and requirements of the underlying business or research project it supports.\n\nIn structured data models, such as the relational model, a schema provides a robust framework for defining:\n* The structure of tables (relations) and their attributes (columns).\n* Rules and constraints that ensure data consistency, accuracy, and reliability.\n* Relationships between tables, such as primary keys (unique identifiers for records) and foreign keys (references to related records in other tables).\n\n### Aims of Good Schema Design\n* Data Integrity: Ensures consistency and prevents anomalies.\n* Query Efficiency: Facilitates fast and accurate data retrieval, supports complex queries, and optimizes database performance.\n* Scalability: Allows the database to grow and adapt as data volumes increase.\n\n### Key Elements of Schema Design\n* Tables and Attributes: Each table is defined with specific attributes (columns), each assigned a data type.\n* Primary Keys: Uniquely identify each record in a table.\n* Foreign Keys: Establish relationships between entities in tables.\n* Indexes: Support efficient queries.\n\nThrough careful schema design, database architects create systems that are both efficient and flexible, meeting the current and future needs of an organization. The schema acts as a living document that guides the structure, operations, and integrity of the database.\n\n## 2. Schema as a Database Module\n\nIn complex database designs, the term \"schema\" is also used to describe a distinct module of a larger database with its own namespace that groups related tables together. \nThis modular approach:\n* Separates tables into logical groups for better organization.\n* Avoids naming conflicts in large databases with multiple schemas."
	`6`	+ "source": "---\ntitle: Schemas\nauthors:\n - name: Dimitri Yatsenko\n---\n\n# What is a schema?\n\nThe term schema has two related meanings in the context of databases:\n\n## 1. Schema as a Data Blueprint\nA schema is a formal specification of the structure of data and the rules governing its integrity.\nIt serves as a blueprint that defines how data is organized, stored, and accessed within a database.\nThis ensures that the database reflects the rules and requirements of the underlying business or research project it supports.\n\nIn structured data models, such as the relational model, a schema provides a robust framework for defining:\n* The structure of tables (relations) and their attributes (columns).\n* Rules and constraints that ensure data consistency, accuracy, and reliability.\n* Relationships between tables, such as primary keys (unique identifiers for records) and foreign keys (references to related records in other tables).\n\n### Aims of Good Schema Design\n* Data Integrity: Ensures consistency and prevents anomalies.\n* Query Efficiency: Facilitates fast and accurate data retrieval, supports complex queries, and optimizes database performance.\n* Scalability: Allows the database to grow and adapt as data volumes increase.\n\n### Key Elements of Schema Design\n* Tables and Attributes: Each table is defined with specific attributes (columns), each assigned a data type.\n* Primary Keys: Uniquely identify each record in a table.\n* Foreign Keys: Establish relationships between entities in tables.\n* Indexes: Support efficient queries.\n\nThrough careful schema design, database architects create systems that are both efficient and flexible, meeting the current and future needs of an organization. The schema acts as a living document that guides the structure, operations, and integrity of the database.\n\n## 2. Schema as a Database Module\n\nIn complex database designs, the term \"schema\" is also used to describe a distinct module of a larger database with its own namespace that groups related tables together. \nThis modular approach:\n* Separates tables into logical groups for better organization.\n* Avoids naming conflicts in large databases with multiple schemas."
`7`	`7`	`},`
`8`	`8`	`{`
`9`	`9`	`"cell_type": "markdown",`