Skip to content

Commit 509ebbd

Browse files
Merge pull request #11 from dimitri-yatsenko/main
Add the Pipeline Projects sections
2 parents 098ac42 + 6569311 commit 509ebbd

File tree

5 files changed

+668
-2269
lines changed

5 files changed

+668
-2269
lines changed

book/20-concepts/00-databases.md

Lines changed: 22 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -13,14 +13,14 @@ The database not only tracks the current state of the enterprise's processes but
1313
**Key traits of databases**:
1414
- Structured data reflects the logic of the enterprise's operations
1515
- Supports the organization's operations by reflecting and enforcing its rules and constraints (data integrity)
16-
- **Precise access control ensures only authorized users can view or modify specific data**
16+
- Precise access control ensures only authorized users can view or modify specific data
1717
- Ability to evolve over time
1818
- Facilitates distributed, concurrent access by multiple users
1919
- Centralized data consistency, appearing as a single source of data even if physically distributed, reflecting all changes
2020
- Allows specific and precise queries through various interfaces for different users
2121
```
2222

23-
Databases are crucial for the smooth and organized operation of various entities, from hotels and airlines to universities, banks, and research projects. They ensure that processes are accurately tracked, essential rules are enforced, only valid transactions are allowed, and **sensitive data is protected** from unauthorized access. This combination of data integrity and data security makes databases indispensable for any operation where data reliability and confidentiality matter.
23+
Databases are crucial for the smooth and organized operation of various entities, from hotels and airlines to universities, banks, and research projects. They ensure that processes are accurately tracked, essential rules are enforced, only valid transactions are allowed, and sensitive data is protected from unauthorized access. This combination of data integrity and data security makes databases indispensable for any operation where data reliability and confidentiality matter.
2424

2525
## Database Management Systems (DBMS)
2626

@@ -29,28 +29,28 @@ A Database Management System (DBMS) is a software system that serves as the comp
2929
It defines and enforces the structure of the data, ensuring that the organization's rules are consistently applied.
3030
A DBMS manages data storage and efficiently executes data updates and queries while safeguarding the data's structure and integrity, particularly in environments with multiple concurrent users.
3131
32-
**Critically, a DBMS also manages user authentication and authorization**, controlling who can access which data and what operations they can perform.
32+
Critically, a DBMS also manages user authentication and authorization, controlling who can access which data and what operations they can perform.
3333
```
3434

3535
Consider an airline's database for flight schedules and ticket bookings. The airline must adhere to several key rules:
3636

3737
* A seat cannot be booked by two passengers for the same flight
3838
* A seat is considered reserved only after all details are verified and payment is processed
39-
* **Only authorized ticketing agents can modify reservations**
40-
* **Passengers can view only their own booking information**
41-
* **Financial data is accessible only to accounting staff**
39+
* Only authorized ticketing agents can modify reservations
40+
* Passengers can view only their own booking information
41+
* Financial data is accessible only to accounting staff
4242

4343
A robust DBMS enforces such rules reliably, ensuring smooth operations while interacting with multiple users and systems at once. The same system that prevents double-booking also prevents unauthorized access to passenger records.
4444

45-
Databases are dynamic, with data continuously updated by both users and systems. Even in the face of disruptions like power outages, errors, or cyberattacks, the DBMS ensures that the system recovers quickly and returns to a stable state. For users, the database should function seamlessly, allowing actions to be performed without interference from others working on the system simultaneously—**while ensuring they can only perform actions they're authorized to do**.
45+
Databases are dynamic, with data continuously updated by both users and systems. Even in the face of disruptions like power outages, errors, or cyberattacks, the DBMS ensures that the system recovers quickly and returns to a stable state. For users, the database should function seamlessly, allowing actions to be performed without interference from others working on the system simultaneously—while ensuring they can only perform actions they're authorized to do.
4646

4747
## Data Security and Access Management
4848

4949
One of the most critical features distinguishing databases from simple file storage is **precise access control**. In scientific research, healthcare, finance, and many other domains, not all data should be accessible to all users.
5050

5151
### Authentication and Authorization
5252

53-
Before you can work with a database, you must **authentication**—prove your identity with a username and password. Once authenticated, the database enforces **authorization** rules that determine what you can do:
53+
Before you can work with a database, you must authenticate—prove your identity with a username and password. Once authenticated, the database enforces authorization rules that determine what you can do:
5454

5555
- **Read**: View specific tables or columns
5656
- **Write**: Add new data to certain tables
@@ -109,10 +109,19 @@ This book focuses on **DataJoint**, a framework that extends relational database
109109
The relational data model—introduced by Edgar F. Codd in 1970—revolutionized data management by organizing data into tables with well-defined relationships. This model has dominated database systems for over five decades due to its mathematical rigor and versatility. Modern relational databases like MySQL and PostgreSQL continue to evolve, incorporating new capabilities for scalability and security while maintaining the core principles that make them reliable and powerful.
110110

111111
The following chapters build the conceptual foundation you need to understand DataJoint's approach:
112-
- **Data Models**: What data models are and why schemas matter for scientific work
113-
- **Relational Theory**: The mathematical foundations that make relational databases powerful
114-
- **Relational Practice**: Hands-on experience with database operations
115-
- **Relational Workflows**: How DataJoint extends relational theory for computational pipelines
116-
- **Scientific Data Pipelines**: How workflows scale into complete research data operations systems
112+
- [Data Models](01-models.md): What data models are and why schemas matter for scientific work
113+
- [Relational Theory](02-relational.md): The mathematical foundations that make relational databases powerful
114+
- [Data Integrity](04-integrity.md): Hands-on experience with database operations
115+
- [Relational Workflows](05-workflows.md): How DataJoint extends relational theory for computational pipelines
116+
- [Scientific Data Pipelines](06-pipelines.md): How workflows scale into complete research data operations systems
117117

118118
By the end, you'll understand both the mathematical foundations and their practical application to your research.
119+
120+
## Links
121+
122+
- [MySQL](https://www.mysql.com/) — Popular open-source relational database management system
123+
- [PostgreSQL](https://www.postgresql.org/) — Advanced open-source relational database
124+
- [SQLite](https://www.sqlite.org/) — Embedded relational database engine
125+
- [Google Spanner](https://cloud.google.com/spanner) — Distributed relational database service
126+
- [CockroachDB](https://www.cockroachlabs.com/) — Distributed SQL database
127+
- [DataJoint](https://datajoint.com/) — Framework for scientific data pipelines

0 commit comments

Comments
 (0)