Implement testcontainers for pytest-managed container lifecycle

Tests now automatically start MySQL and MinIO containers via testcontainers.
No manual `docker-compose up` required - just run `pytest tests/`.

Changes:
- conftest.py: Add mysql_container and minio_container fixtures that
  auto-start containers when tests run and stop them afterward
- pyproject.toml: Add testcontainers[mysql,minio] dependency, update
  pixi tasks, remove pytest-env (no longer needed)
- docker-compose.yaml: Update docs to clarify it's optional for tests
- README.md: Comprehensive developer guide with clear instructions for
  running tests, pre-commit hooks, and PR submission checklist

Usage:
- Default: `pytest tests/` - testcontainers manages containers
- External: `DJ_USE_EXTERNAL_CONTAINERS=1 pytest` - use docker-compose

Benefits:
- Zero setup for developers - just `pip install -e ".[test]" && pytest`
- Dynamic ports (no conflicts with other services)
- Automatic cleanup after tests
- Simpler CI configuration

Version bump to 2.0.0a7

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>

Author: dimitri-yatsenko

README.md

@@ -146,30 +146,57 @@ DataJoint (<https://datajoint.com>).
 
 ### Prerequisites
 
-- [Docker](https://docs.docker.com/get-docker/) for MySQL and MinIO services
+- [Docker](https://docs.docker.com/get-docker/) (Docker daemon must be running)
 - Python 3.10+
 
-### Running Tests
-
-Tests are organized into `unit/` (no external services) and `integration/` (requires MySQL + MinIO):
+### Quick Start
 
 ```bash
-# Install dependencies
+# Clone and install
+git clone https://github.com/datajoint/datajoint-python.git
+cd datajoint-python
 pip install -e ".[test]"
 
-# Run unit tests only (fast, no Docker needed)
-pytest tests/unit/
+# Run all tests (containers start automatically via testcontainers)
+pytest tests/
 
-# Start MySQL and MinIO for integration tests
-docker compose up -d db minio
+# Install and run pre-commit hooks
+pip install pre-commit
+pre-commit install
+pre-commit run --all-files
+```
 
-# Run all tests
+### Running Tests
+
+Tests use [testcontainers](https://testcontainers.com/) to automatically manage MySQL and MinIO containers.
+**No manual `docker-compose up` required** - containers start when tests run and stop afterward.
+
+```bash
+# Run all tests (recommended)
 pytest tests/
 
+# Run with coverage report
+pytest --cov-report term-missing --cov=datajoint tests/
+
 # Run specific test file
 pytest tests/integration/test_blob.py -v
 
-# Stop services when done
+# Run only unit tests (no containers needed)
+pytest tests/unit/
+```
+
+### Alternative: External Containers
+
+For development/debugging, you may prefer persistent containers that survive test runs:
+
+```bash
+# Start containers manually
+docker compose up -d db minio
+
+# Run tests using external containers
+DJ_USE_EXTERNAL_CONTAINERS=1 pytest tests/
+
+# Stop containers when done
 docker compose down
 ```
 
@@ -183,24 +210,46 @@ docker compose --profile test up djtest --build
 
 ### Alternative: Using pixi
 
-[pixi](https://pixi.sh) users can run tests with automatic service management:
+[pixi](https://pixi.sh) users can run tests with:
 
 ```bash
 pixi install        # First time setup
-pixi run test       # Starts services and runs tests
-pixi run services-down  # Stop services
+pixi run test       # Runs tests (testcontainers manages containers)
 ```
 
 ### Pre-commit Hooks
 
+Pre-commit hooks run automatically on `git commit` to check code quality.
+**All hooks must pass before committing.**
+
 ```bash
-pre-commit install          # Install hooks (first time)
-pre-commit run --all-files  # Run all checks
+# Install hooks (first time only)
+pip install pre-commit
+pre-commit install
+
+# Run all checks manually
+pre-commit run --all-files
+
+# Run specific hook
+pre-commit run ruff --all-files
+pre-commit run codespell --all-files
 ```
 
+Hooks include:
+- **ruff**: Python linting and formatting
+- **codespell**: Spell checking
+- **YAML/JSON/TOML validation**
+- **Large file detection**
+
+### Before Submitting a PR
+
+1. **Run all tests**: `pytest tests/`
+2. **Run pre-commit**: `pre-commit run --all-files`
+3. **Check coverage**: `pytest --cov-report term-missing --cov=datajoint tests/`
+
 ### Environment Variables
 
-Tests use these defaults (configured in `pyproject.toml`):
+For external container mode (`DJ_USE_EXTERNAL_CONTAINERS=1`):
 
 | Variable | Default | Description |
 |----------|---------|-------------|

docker-compose.yaml

@@ -1,10 +1,15 @@
 # Development environment with MySQL and MinIO services
 #
-# Quick start:
-#   docker compose up -d db minio   # Start services
-#   pytest tests/                    # Run tests (uses localhost defaults)
+# NOTE: docker-compose is OPTIONAL for running tests.
+# Tests use testcontainers to automatically manage containers.
+# Just run: pytest tests/
 #
-# Full Docker testing:
+# Use docker-compose for development/debugging when you want
+# persistent containers that survive test runs:
+#   docker compose up -d db minio   # Start services manually
+#   pytest tests/                    # Tests will use these containers
+#
+# Full Docker testing (CI):
 #   docker compose --profile test up djtest --build
 services:
   db:

pyproject.toml

@@ -84,9 +84,9 @@ datajoint = "datajoint.cli:cli"
 test = [
   "pytest",
   "pytest-cov",
-  "pytest-env",
   "requests",
   "graphviz",
+  "testcontainers[mysql,minio]>=4.0",
 ]
 
 [project.optional-dependencies]
@@ -96,9 +96,9 @@ azure = ["adlfs>=2023.1.0"]
 test = [
   "pytest",
   "pytest-cov",
-  "pytest-env",
   "requests",
   "s3fs>=2023.1.0",
+  "testcontainers[mysql,minio]>=4.0",
 ]
 dev = [
   "pre-commit",
@@ -158,20 +158,7 @@ skip = ".git,*.pdf,*.svg,*.csv,*.ipynb,*.drawio"
 # astroid -- Python library name (not "asteroid")
 ignore-words-list = "rever,numer,astroid"
 
-[tool.pytest_env]
-# Default environment variables for tests (D: prefix = only set if not defined)
-# These defaults work for local development with `docker compose up -d db minio`
-# For devcontainer/docker: override DJ_HOST=db and S3_ENDPOINT=minio:9000
-"D:DJ_HOST" = "localhost"
-"D:DJ_PORT" = "3306"
-"D:DJ_USER" = "root"
-"D:DJ_PASS" = "password"
-"D:DJ_TEST_USER" = "datajoint"
-"D:DJ_TEST_PASSWORD" = "datajoint"
-"D:S3_ENDPOINT" = "localhost:9000"
-"D:S3_ACCESS_KEY" = "datajoint"
-"D:S3_SECRET_KEY" = "datajoint"
-"D:S3_BUCKET" = "datajoint.test"
+# pytest-env removed - testcontainers handles container lifecycle automatically
 
 
 [tool.pixi.workspace]
@@ -187,12 +174,13 @@ dev = { features = ["dev"], solve-group = "default" }
 test = { features = ["test"], solve-group = "default" }
 
 [tool.pixi.tasks]
-# Start required services (MySQL and MinIO)
+# Tests use testcontainers - no manual setup required
+test = "pytest tests/"
+test-cov = "pytest --cov-report term-missing --cov=datajoint tests/"
+# Optional: use external containers (docker-compose) instead of testcontainers
 services-up = "docker compose up -d db minio"
 services-down = "docker compose down"
-# Run tests (requires services to be running, uses localhost defaults from pytest_env)
-test = { cmd = "pytest tests/", depends-on = ["services-up"] }
-test-cov = { cmd = "pytest --cov-report term-missing --cov=datajoint tests/", depends-on = ["services-up"] }
+test-external = { cmd = "DJ_USE_EXTERNAL_CONTAINERS=1 pytest tests/", depends-on = ["services-up"] }
 
 [tool.pixi.dependencies]
 python = ">=3.10,<3.14"

src/datajoint/version.py

@@ -1,4 +1,4 @@
 # version bump auto managed by Github Actions:
 # label_prs.yaml(prep), release.yaml(bump), post_release.yaml(edit)
 # manually set this version will be eventually overwritten by the above actions
-__version__ = "2.0.0a6"
+__version__ = "2.0.0a7"