Skip to content

Commit 0d7e6c5

Browse files
docs: landing page and README improvements (datajoint#105)
* docs: remove 'breaking' from landing page announcement Changed 'major breaking release' to 'major release' * docs: update migration guide links in README - Changed migrate-from-0x.md to migrate-to-v20.md (2 references) - Ensures README points to correct migration guide file * docs: simplify README - remove redundancy, reference live docs - Removed Quick Start section (covered on docs.datajoint.com) - Condensed local development instructions - Simplified Contributing, Related, Citation sections - Reduced from ~310 lines to ~87 lines - Focus README on repo structure and contributing, not usage docs * docs: add license history note (LGPLv2.1 prior to v2.0) * docs: address reviewer feedback on README - Separate git clone commands for clarity (line 30-31) - Add BUILD mode example comment (line 37-38) - Add Windows venv activation comment (line 51) - Use clearer placeholder credentials (line 56-57) - Add chmod 600 for secrets security (line 58) - Fix citation link trailing slash consistency (line 86)
1 parent 6d7bf91 commit 0d7e6c5

File tree

2 files changed

+36
-255
lines changed

2 files changed

+36
-255
lines changed

README.md

Lines changed: 35 additions & 254 deletions
Original file line numberDiff line numberDiff line change
@@ -1,310 +1,91 @@
11
# DataJoint Documentation
22

3-
Official documentation for [DataJoint](https://github.com/datajoint/datajoint-python) 2.0,
4-
an open-source framework for building scientific data pipelines.
3+
Official documentation for [DataJoint](https://github.com/datajoint/datajoint-python) 2.0 — an open-source framework for building scientific data pipelines.
54

6-
**Live site:** https://docs.datajoint.com
5+
**📖 Live site:** https://docs.datajoint.com
76

8-
> **📘 Upgrading from legacy DataJoint (pre-2.0)?**
9-
> See the **[Migration Guide](https://docs.datajoint.com/how-to/migrate-from-0x/)** for a step-by-step upgrade path.
7+
> **Upgrading from pre-2.0?** See the [Migration Guide](https://docs.datajoint.com/how-to/migrate-to-v20/)
108
11-
## What is DataJoint?
9+
## About DataJoint
1210

13-
DataJoint is a Python framework for building scientific data pipelines using relational
14-
databases. It implements the **Relational Workflow Model**—a paradigm that extends
15-
relational databases with native support for computational workflows.
16-
17-
Key features:
18-
19-
- **Declarative schema design** — Define tables and relationships in Python
20-
- **Automatic dependency tracking** — Foreign keys encode workflow dependencies
21-
- **Built-in computation** — Imported and Computed tables run automatically
22-
- **Data integrity** — Referential integrity and transaction support
23-
- **Reproducibility** — Immutable data with full provenance
24-
25-
## Quick Start
26-
27-
### Installation
28-
29-
```bash
30-
pip install datajoint
31-
```
32-
33-
For schema diagrams, install Graphviz (the system library, not just Python bindings):
34-
35-
```bash
36-
# macOS
37-
brew install graphviz
38-
39-
# Ubuntu/Debian
40-
sudo apt-get install graphviz libgraphviz-dev
41-
42-
# conda (any platform)
43-
conda install -c conda-forge graphviz pygraphviz
44-
```
45-
46-
If using pip (after installing system Graphviz):
47-
```bash
48-
pip install pygraphviz
49-
```
50-
51-
### Configuration
52-
53-
DataJoint uses configuration files to manage database credentials securely. Create these
54-
files in your project directory:
55-
56-
**datajoint.json** (non-sensitive settings, commit to version control):
57-
```json
58-
{
59-
"database": {
60-
"host": "localhost",
61-
"port": 3306
62-
}
63-
}
64-
```
65-
66-
**.secrets/database.user** and **.secrets/database.password** (sensitive, add to .gitignore):
67-
```bash
68-
mkdir -p .secrets
69-
echo "your_username" > .secrets/database.user
70-
echo "your_password" > .secrets/database.password
71-
chmod 600 .secrets/*
72-
echo ".secrets/" >> .gitignore
73-
```
74-
75-
DataJoint automatically discovers these files by searching up from the current directory.
76-
This keeps credentials out of your code and version control.
77-
78-
### Define a Schema
79-
80-
```python
81-
import datajoint as dj
82-
83-
schema = dj.Schema('my_pipeline')
84-
85-
@schema
86-
class Subject(dj.Manual):
87-
definition = """
88-
subject_id : int32
89-
---
90-
name : varchar(100)
91-
date_of_birth : date
92-
"""
93-
94-
@schema
95-
class Session(dj.Manual):
96-
definition = """
97-
-> Subject
98-
session_idx : int32
99-
---
100-
session_date : date
101-
duration : float32 # minutes
102-
notes = '' : varchar(1000)
103-
"""
104-
105-
@schema
106-
class ProcessedData(dj.Computed):
107-
definition = """
108-
-> Session
109-
---
110-
result : float64
111-
"""
112-
113-
def make(self, key):
114-
# Compute result from session data
115-
duration = (Session & key).fetch1('duration')
116-
self.insert1({**key, 'result': duration * 2})
117-
```
118-
119-
Note: Use DataJoint core types (`int32`, `float32`, `float64`, `varchar`) for portability
120-
across database backends.
121-
122-
### View Schema Diagram
123-
124-
```python
125-
dj.Diagram(schema)
126-
```
127-
128-
### Run Computations
129-
130-
```python
131-
ProcessedData.populate()
132-
```
11+
DataJoint is a Python framework for scientific data pipelines built on the **Relational Workflow Model**. For installation, tutorials, and complete documentation, visit **[docs.datajoint.com](https://docs.datajoint.com)**.
13312

13413
## Documentation Structure
13514

136-
This documentation follows the [Diátaxis](https://diataxis.fr/) framework:
137-
138-
| Section | Purpose | Link |
139-
|---------|---------|------|
140-
| **Tutorials** | Learn by building real pipelines | [src/tutorials/](src/tutorials/) |
141-
| **How-To Guides** | Practical guides for common tasks | [src/how-to/](src/how-to/) |
142-
| **Explanation** | Understand the principles behind DataJoint | [src/explanation/](src/explanation/) |
143-
| **Reference** | Specifications and API documentation | [src/reference/](src/reference/) |
15+
This repository contains the source for the DataJoint documentation, organized using the [Diátaxis](https://diataxis.fr/) framework:
14416

145-
Key pages:
146-
- **[Migration Guide](src/how-to/migrate-from-0x.md)** — Upgrade from legacy DataJoint (pre-2.0)
147-
- **[What's New in 2.0](src/explanation/whats-new-2.md)** — Major changes and improvements
17+
- **[Tutorials](https://docs.datajoint.com/tutorials/)** — Learn by building real pipelines
18+
- **[How-To Guides](https://docs.datajoint.com/how-to/)** — Practical task-oriented guides
19+
- **[Explanation](https://docs.datajoint.com/explanation/)** — Understanding concepts and design
20+
- **[Reference](https://docs.datajoint.com/reference/)** — Specifications and API documentation
14821

149-
## Local Development with Docker (Recommended)
22+
## Local Development
15023

151-
The Docker environment includes MySQL, MinIO (S3-compatible storage), Graphviz, and all
152-
dependencies needed to build documentation and execute tutorial notebooks.
153-
154-
### Start the Environment
24+
### Docker (Recommended)
15525

15626
```bash
157-
# Clone the documentation repository
27+
# Clone repositories
15828
git clone https://github.com/datajoint/datajoint-docs.git
15929
cd datajoint-docs
160-
161-
# Clone datajoint-python pre-release branch (required for API docs)
16230
cd ..
16331
git clone -b pre/v2.0 https://github.com/datajoint/datajoint-python.git
16432
cd datajoint-docs
16533

166-
# Start all services (MySQL, MinIO, docs server)
34+
# Start live preview at http://localhost:8000
16735
MODE="LIVE" docker compose up --build
168-
```
169-
170-
Navigate to http://127.0.0.1:8000/
171-
172-
### Services
173-
174-
| Service | Port | Description |
175-
|---------|------|-------------|
176-
| `docs` | 8000 | MkDocs live server |
177-
| `mysql` | 3306 | MySQL 8.0 database |
178-
| `minio` | 9002 | MinIO S3 API |
179-
| `minio` | 9003 | MinIO console |
180-
181-
### Execute Tutorial Notebooks
182-
183-
Tutorial notebooks can be executed inside the Docker environment where the database
184-
is available:
18536

186-
```bash
187-
# Execute a single notebook
188-
docker compose exec docs jupyter nbconvert \
189-
--to notebook --execute --inplace \
190-
/main/src/tutorials/01-getting-started.ipynb
191-
192-
# Execute all tutorials
193-
docker compose exec docs bash -c '
194-
for nb in /main/src/tutorials/*.ipynb; do
195-
jupyter nbconvert --to notebook --execute --inplace "$nb"
196-
done
197-
'
37+
# Build static site (optional)
38+
# MODE="BUILD" docker compose up --build
19839
```
19940

200-
### Build Static Site
41+
The Docker environment includes MySQL, MinIO, and all dependencies.
20142

202-
```bash
203-
# Build static HTML (output in site/)
204-
MODE="BUILD" docker compose up --build
205-
```
43+
### Native Python
20644

207-
### Reset Database
45+
**Prerequisites:** Python 3.10+, MySQL 8.0+
20846

20947
```bash
210-
# Stop services and remove data volumes
211-
docker compose down -v
212-
```
213-
214-
## Local Development without Docker
215-
216-
### Prerequisites
217-
218-
- Python 3.10+
219-
- MySQL 8.0+ (running locally)
220-
- Graphviz (for schema diagrams)
221-
222-
### Setup
223-
224-
```bash
225-
# Clone the repository
48+
# Setup
22649
git clone https://github.com/datajoint/datajoint-docs.git
22750
cd datajoint-docs
228-
229-
# Create virtual environment
230-
python -m venv .venv
231-
source .venv/bin/activate # or .venv\Scripts\activate on Windows
232-
233-
# Install dependencies
51+
python -m venv .venv && source .venv/bin/activate # Windows: .venv\Scripts\activate
23452
pip install -r pip_requirements.txt
235-
```
236-
237-
Note: For schema diagrams, ensure Graphviz system libraries are installed (see Quick Start).
23853

239-
### Configure Database Connection
240-
241-
The repository includes a `datajoint.json` with default settings. Create the secrets
242-
directory with your credentials:
243-
244-
```bash
54+
# Configure credentials
24555
mkdir -p .secrets
24656
echo "your_username" > .secrets/database.user
24757
echo "your_password" > .secrets/database.password
24858
chmod 600 .secrets/*
249-
```
250-
251-
### Preview Documentation
25259

253-
```bash
60+
# Start live preview at http://localhost:8000
25461
mkdocs serve
25562
```
25663

257-
Navigate to http://127.0.0.1:8000/
258-
25964
## Contributing
26065

261-
Contributions are welcome! See our [contribution guidelines](src/about/contributing.md).
262-
263-
### Quick Fixes
264-
265-
1. Fork the repository
266-
2. Edit the relevant markdown file in `src/`
267-
3. Submit a pull request
268-
269-
### Larger Changes
66+
Contributions welcome! See [contribution guidelines](https://docs.datajoint.com/about/contributing/).
27067

271-
1. Open an issue to discuss the change
272-
2. Fork and create a feature branch
273-
3. Make changes with `mkdocs serve` for preview
274-
4. Submit a pull request
275-
276-
### Executing Notebooks for CI
277-
278-
When modifying tutorial notebooks, re-execute them to update outputs:
68+
**Quick fixes:** Fork, edit markdown in `src/`, submit PR.
27969

70+
**Tutorial notebooks:** Re-execute after changes:
28071
```bash
281-
docker compose exec docs jupyter nbconvert \
282-
--to notebook --execute --inplace \
283-
--ExecutePreprocessor.timeout=300 \
72+
docker compose exec docs jupyter nbconvert --to notebook --execute --inplace \
28473
/main/src/tutorials/YOUR_NOTEBOOK.ipynb
28574
```
28675

287-
## Related Repositories
76+
## Related
28877

289-
- **[datajoint-python](https://github.com/datajoint/datajoint-python)** — Core DataJoint library
290-
- **[DataJoint Elements](https://datajoint.com/docs/elements/)** — Neuroscience pipeline elements
291-
- **[DataJoint Works](https://datajoint.com)**Company and commercial support
78+
- [datajoint-python](https://github.com/datajoint/datajoint-python) — Core library
79+
- [DataJoint Elements](https://docs.datajoint.com/elements/) — Neuroscience pipeline modules
80+
- [datajoint.com](https://datajoint.com)Commercial support
29281

29382
## Citation
29483

295-
If you use DataJoint in your research, please cite:
296-
297-
> Yatsenko D, Walker EY, Tolias AS. DataJoint: A Simpler Relational Data Model.
298-
> arXiv:2303.00102. 2023. doi: [10.48550/arXiv.2303.00102](https://doi.org/10.48550/arXiv.2303.00102)
84+
> Yatsenko D, Walker EY, Tolias AS. DataJoint: A Simpler Relational Data Model. arXiv:2303.00102. 2023. doi: [10.48550/arXiv.2303.00102](https://doi.org/10.48550/arXiv.2303.00102)
29985
300-
Earlier publication:
301-
302-
> Yatsenko D, Reimer J, Ecker AS, Walker EY, Sinz F, Berens P, Hoenselaar A,
303-
> Cotton RJ, Siapas AS, Tolias AS. DataJoint: managing big scientific data
304-
> using MATLAB or Python. bioRxiv. 2015. doi: [10.1101/031658](https://doi.org/10.1101/031658)
86+
Full citation information: [docs.datajoint.com/about/citation/](https://docs.datajoint.com/about/citation/)
30587

30688
## License
30789

308-
Documentation: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)
309-
310-
DataJoint software: [Apache 2.0](https://github.com/datajoint/datajoint-python/blob/master/LICENSE)
90+
- Documentation: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)
91+
- DataJoint software: [Apache 2.0](https://github.com/datajoint/datajoint-python/blob/master/LICENSE) (LGPLv2.1 prior to v2.0)

src/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# DataJoint Documentation
22

3-
> **DataJoint 2.0 is a major breaking release.** Existing pipelines require migration.
3+
> **DataJoint 2.0 is a major release.** Existing pipelines require migration.
44
> See the [Migration Guide](how-to/migrate-to-v20.md) for upgrade instructions.
55
> For pre-2.0 documentation, visit [datajoint.github.io/datajoint-python](https://datajoint.github.io/datajoint-python).
66

0 commit comments

Comments
 (0)