Improve development documentation and scripts

This commit enhances the developer experience by:

- Updating the main README.md with a "Getting Started / Local Development Setup" section, providing clear instructions for running your project locally using Docker Compose.
- Adding a "Useful Scripts" section to the main README.md, documenting key scripts for building, testing, linting, formatting, and client generation.
- Reviewing and updating `development.md` to ensure clarity on Docker Compose usage, local development workflows, and service URLs.
- Reviewing and updating `backend/README.md` and `frontend/README.md` to ensure accuracy of setup, development, and testing instructions.
- Reviewing and adding comments to various scripts in `scripts/` and `backend/scripts/` for better understanding and maintainability.
- Ensuring consistency in commands (e.g., `fastapi dev` vs `fastapi run --reload`).

These changes aim to make it easier for new developers to get started with your project and for existing developers to utilize available scripts and documentation effectively.

Author: google-labs-jules[bot]

README.md

@@ -212,6 +212,16 @@ The input variables, with their default values (some auto generated) are:
 - `postgres_password`: (default: `"changethis"`) The password for the PostgreSQL database, stored in .env, you can generate one with the method above.
 - `sentry_dsn`: (default: "") The DSN for Sentry, if you are using it, you can set it later in .env.
 
+## Getting Started / Local Development Setup
+
+Docker Compose is the recommended way to run the project locally.
+Use the command `docker compose watch` to start the services.
+
+For more details on Docker Compose, including how to access services (frontend, backend, docs, etc.), please refer to `development.md`.
+
+For faster development iteration, frontend and backend services can be run directly on the host machine.
+Instructions for this can be found in `frontend/README.md` and `backend/README.md`.
+
 ## Backend Development
 
 Backend docs: [backend/README.md](./backend/README.md).
@@ -230,6 +240,18 @@ General development docs: [development.md](./development.md).
 
 This includes using Docker Compose, custom local domains, `.env` configurations, etc.
 
+## Useful Scripts
+
+Here's a list of scripts available in the project to help with common development tasks:
+
+-   `scripts/build.sh`: Builds the Docker images for the project.
+-   `scripts/test.sh`: Runs the complete test suite in a Dockerized environment. This typically includes backend tests and can be expanded to include frontend end-to-end tests.
+-   `scripts/test-local.sh`: Runs backend tests directly on the host. It assumes the backend services (like the database) are already running (e.g., via `docker compose watch` or a similar local setup).
+-   `scripts/generate-client.sh`: Generates or updates the frontend client based on the backend's OpenAPI schema. This usually involves fetching the schema and running a code generation tool.
+-   `backend/scripts/format.sh`: Formats the backend Python code using Ruff to ensure consistent code style.
+-   `backend/scripts/lint.sh`: Lints the backend Python code using MyPy for static type checking and Ruff for identifying potential errors and style issues.
+-   `backend/scripts/test.sh`: Runs backend tests directly on the host (similar to `scripts/test-local.sh` but often focused only on backend unit/integration tests) and generates a test coverage report.
+
 ## Release Notes
 
 Check the file [release-notes.md](./release-notes.md).

backend/README.md

@@ -71,16 +71,16 @@ root@7f2607af31c3:/app#
 
 that means that you are in a `bash` session inside your container, as a `root` user, under the `/app` directory, this directory has another directory called "app" inside, that's where your code lives inside the container: `/app/app`.
 
-There you can use the `fastapi run --reload` command to run the debug live reloading server.
+There you can use the `fastapi dev` command to run the debug live reloading server.
 
 ```console
-$ fastapi run --reload app/main.py
+$ fastapi dev app/main.py
 ```
 
 ...it will look like:
 
 ```console
-root@7f2607af31c3:/app# fastapi run --reload app/main.py
+root@7f2607af31c3:/app# fastapi dev app/main.py
 ```
 
 and then hit enter. That runs the live reloading server that auto reloads when it detects code changes.

backend/scripts/format.sh

@@ -1,5 +1,8 @@
 #!/bin/sh -e
+# Print commands and their arguments as they are executed
 set -x
 
+# Run ruff to check for linting issues in 'app' and 'scripts' directories and automatically fix them.
 ruff check app scripts --fix
+# Run ruff to format the code in 'app' and 'scripts' directories.
 ruff format app scripts

backend/scripts/lint.sh

@@ -1,8 +1,16 @@
 #!/usr/bin/env bash
 
+# Exit in case of error
 set -e
+# Print commands and their arguments as they are executed
 set -x
 
+# Run Mypy for static type checking on the 'app' directory.
 mypy app
+# Run Ruff to check for linting errors in the 'app' directory.
+# This command only reports errors, it does not fix them.
 ruff check app
+# Run Ruff formatter in check mode on the 'app' directory.
+# This command reports files that need formatting without actually modifying them.
+# It's useful for CI to verify that code is correctly formatted.
 ruff format app --check

backend/scripts/test.sh

@@ -1,8 +1,23 @@
 #!/usr/bin/env bash
 
+# Exit in case of error
 set -e
+# Print commands and their arguments as they are executed
 set -x
 
+# Run pytest with coverage measurement.
+# --source=app specifies that coverage should be measured for the 'app' directory.
+# -m pytest ensures that pytest is run as a module.
 coverage run --source=app -m pytest
+
+# Generate a text-based coverage report in the terminal.
+# --show-missing will also list the line numbers of code that were not executed.
 coverage report --show-missing
+
+# Generate an HTML coverage report.
+# The output will be in a directory named 'htmlcov' by default.
+# The --title option sets the title of the HTML report.
+# "${@-coverage}" attempts to use any arguments passed to this script in the title.
+# For example, if the script is called with `backend/scripts/test.sh -k my_test`,
+# the title might become "-k my_test-coverage". If no arguments are passed, it will be "-coverage".
 coverage html --title "${@-coverage}"

development.md

@@ -16,6 +16,8 @@ Backend, JSON based web API based on OpenAPI: http://localhost:8000
 
 Automatic interactive documentation with Swagger UI (from the OpenAPI backend): http://localhost:8000/docs
 
+Automatic alternative documentation with ReDoc (from the OpenAPI backend): http://localhost:8000/redoc
+
 Adminer, database web administration: http://localhost:8080
 
 Traefik UI, to see how the routes are being handled by the proxy: http://localhost:8090

scripts/build.sh

@@ -3,8 +3,14 @@
 # Exit in case of error
 set -e
 
+# Set the TAG for Docker images, and exit if not provided
+# TAG is typically the version or a git commit hash
 TAG=${TAG?Variable not set} \
+# Set the FRONTEND_ENV to 'production' if not already set
+# This ensures the frontend is built with production optimizations
 FRONTEND_ENV=${FRONTEND_ENV-production} \
+# Build the Docker images using docker-compose
+# The -f flag specifies the docker-compose file to use
 docker-compose \
 -f docker-compose.yml \
 build

scripts/generate-client.sh

@@ -1,12 +1,29 @@
 #! /usr/bin/env bash
 
+# Exit in case of error
 set -e
+# Print commands and their arguments as they are executed
 set -x
 
+# Navigate to the backend directory
 cd backend
+# Generate the OpenAPI schema from the FastAPI application
+# This command executes a Python one-liner:
+# - Imports the main FastAPI application (app.main)
+# - Imports the json library
+# - Accesses the app's OpenAPI schema (app.main.app.openapi())
+# - Dumps the schema to a JSON string
+# - Redirects the output to openapi.json in the parent (project root) directory
+# Note: This step assumes that the necessary Python environment is active
+# or that the FastAPI application and its dependencies can be imported.
 python -c "import app.main; import json; print(json.dumps(app.main.app.openapi()))" > ../openapi.json
+# Navigate back to the project root directory
 cd ..
+# Move the generated openapi.json to the frontend directory
 mv openapi.json frontend/
+# Navigate to the frontend directory
 cd frontend
+# Run the npm script to generate the API client code using the openapi.json
 npm run generate-client
+# Format the generated client code using Biome
 npx biome format --write ./src/client

scripts/test-local.sh

@@ -1,15 +1,31 @@
 #! /usr/bin/env bash
 
+#! /usr/bin/env bash
+
 # Exit in case of error
 set -e
 
+# Stop and remove any existing containers, volumes, and orphaned containers
+# This ensures a clean environment before starting the tests
 docker-compose down -v --remove-orphans # Remove possibly previous broken stacks left hanging after an error
 
+# On Linux systems, remove __pycache__ directories.
+# Note: Using sudo here might indicate a permissions issue if these files are not owned by the current user.
+# __pycache__ directories are created by Python and generally should not require sudo for removal.
 if [ $(uname -s) = "Linux" ]; then
     echo "Remove __pycache__ files"
     sudo find . -type d -name __pycache__ -exec rm -r {} \+
 fi
 
+# Build the Docker images for all services
 docker-compose build
+# Start all services in detached mode (in the background)
 docker-compose up -d
+# Execute the backend test script (scripts/tests-start.sh) inside the 'backend' container
+# -T disables pseudo-TTY allocation, suitable for non-interactive scripts
+# "$@" forwards all arguments passed to this script to the tests-start.sh script
 docker-compose exec -T backend bash scripts/tests-start.sh "$@"
+
+# Note: Unlike test.sh, this script does not run `docker-compose down` at the end.
+# This means the services will remain running after the tests complete,
+# potentially for inspection or further local development.

scripts/test.sh

@@ -2,10 +2,23 @@
 
 # Exit in case of error
 set -e
+# Print commands and their arguments as they are executed
 set -x
 
+# Build the Docker images for all services
 docker compose build
+
+# Stop and remove any existing containers, volumes, and orphaned containers
+# This ensures a clean environment before starting the tests
 docker compose down -v --remove-orphans # Remove possibly previous broken stacks left hanging after an error
+
+# Start all services in detached mode (in the background)
 docker compose up -d
+
+# Execute the backend test script (scripts/tests-start.sh) inside the 'backend' container
+# -T disables pseudo-TTY allocation, suitable for non-interactive scripts
+# "$@" forwards all arguments passed to this script to the tests-start.sh script
 docker compose exec -T backend bash scripts/tests-start.sh "$@"
+
+# Stop and remove all containers, volumes, and orphaned containers after tests are done
 docker compose down -v --remove-orphans