Add Docker Compose and Dev Containers

Can now develop without having to set up postgres yourself

Author: jarofgreen

.devcontainer.json

@@ -0,0 +1,12 @@
+{
+	"name": "OC4IDS Datastore Pipeline Dev Container",
+	"dockerComposeFile": "docker-compose.dev.yml",
+	"service": "app",
+	"workspaceFolder": "/oc4ids_datastore_pipeline",
+	"forwardPorts": [5432],
+	"customizations": {
+		"vscode": {
+			"extensions":["ms-python.python"]
+		}
+	}
+}

Dockerfile.dev

@@ -0,0 +1,16 @@
+FROM python:3.12-slim
+
+RUN apt-get update \
+  && apt-get install -y libpq-dev gcc postgresql-client
+
+WORKDIR /oc4ids_datastore_pipeline
+
+COPY requirements.txt .
+
+RUN pip install -r requirements.txt
+
+COPY . .
+
+RUN pip install -e .
+
+ENTRYPOINT ["sleep", "infinity"]

README.md

@@ -2,25 +2,16 @@
 
 A Python application to validate and store published OC4IDS datasets.
 
-## Local Development
+## Local Development via Dev Containers or Docker Compose
 
-### Prerequisites
+You can open this repository in a dev container to get an environment complete with Postgres database.
 
-- Python 3.12
-- Postgres
-
-### Install Python requirements
-
-```
-python -m venv .venv
-source .venv/bin/activate
-pip install -r requirements_dev.txt
-```
-
-### Set database enrivonment variable
+If you prefer to use Docker Compose, you can instead run:
 
 ```
-export DATABASE_URL="postgresql://oc4ids_datastore@localhost/oc4ids_datastore"
+docker compose -f docker-compose.dev.yml up -d
+docker compose -f docker-compose.dev.yml exec app bash
+docker compose -f docker-compose.dev.yml stop
 ```
 
 ### Run database migrations
@@ -89,9 +80,19 @@ To send failure notifications by email, the following environment variables must
 ### Run app
 
 ```
-pip install -e .
 oc4ids-datastore-pipeline
 ```
+### Access Database
+
+From inside the dev container or Docker container:
+
+```
+psql postgresql://postgres:postgres@localhost:5432/postgres
+```
+
+Connecting from outside:
+* If using a dev container or Docker Compose locally the same command should work
+* In GitHub Codespaces, we're not sure how to access the port
 
 ### Run linting and type checking
 

docker-compose.dev.yml

@@ -0,0 +1,28 @@
+services:
+  app:
+    build:
+      context: .
+      dockerfile: Dockerfile.dev
+    volumes:
+      - .:/oc4ids_datastore_pipeline:cached
+    # Runs app on the same network as the database container, allows "forwardPorts" in devcontainer.json function.
+    network_mode: service:db
+    environment:
+      - DATABASE_URL=postgresql://postgres:postgres@localhost/postgres
+    entrypoint: sleep infinity
+
+  db:
+    image: postgres:latest
+    restart: unless-stopped
+    volumes:
+      - postgres-data:/var/lib/postgresql/data
+    environment:
+      POSTGRES_USER: postgres
+      POSTGRES_DB: postgres
+      POSTGRES_PASSWORD: postgres
+    # Forwarding this port so people can access the DB when run via Docker Compose without dev containers.
+    ports:
+      - 5432:5432
+
+volumes:
+  postgres-data: