diff --git a/README.md b/README.md index 40d681604..23d9ed5a4 100644 --- a/README.md +++ b/README.md @@ -77,7 +77,7 @@ A sample environment file is provided at the repository root. Create a working e cp sample.env .env ``` -Edit the `.env` file if adjustments are required for the local setup. Key tenancy options (`M8FLOW_DEFAULT_TENANT_ID`, `M8FLOW_ALLOW_MISSING_TENANT_CONTEXT`, `M8FLOW_TENANT_CLAIM`) are described in `sample.env`. +Edit the `.env` file if adjustments are required for the local setup. **Environment variable meanings and examples** are documented in [docs/env-reference.md](docs/env-reference.md) (canonical reference; avoid duplicating long env explanations in this README). Variable names for tenancy and other areas also appear in `sample.env`. --- @@ -217,7 +217,7 @@ After the containers start, continue below to the Keycloak Setup to import the r ### Import Realm -You can import realms manually as below, or run `./extensions/m8flow-backend/keycloak/start_keycloak.sh` after starting Docker to import the identity realm and tenant-a. +You can import realms by running `./extensions/m8flow-backend/keycloak/start_keycloak.sh` after starting Docker to import the identity realm. In the Keycloak Admin Console http://localhost:7002/ log in using the configured administrator credentials. @@ -232,6 +232,8 @@ Browse or copy the content of `extensions/m8flow-backend/keycloak/realm_exports/ +For Keycloak-related URL and issuer settings in `.env`, see [docs/env-reference.md](docs/env-reference.md) (Keycloak URLs). + ### Configure Client With the realm "spiffworkflow" selected, click on "Clients" and then on the client ID **spiffworkflow-backend**. diff --git a/docker/DEPLOYMENT.md b/docker/DEPLOYMENT.md index 7e489c8cf..f11b41bd4 100644 --- a/docker/DEPLOYMENT.md +++ b/docker/DEPLOYMENT.md @@ -37,7 +37,7 @@ export TAG="latest" Ensure the image is built from the repo that includes the m8flow-backend extension (tenancy, Keycloak realm APIs, and support for `SPIFFWORKFLOW_BACKEND_WSGI_PATH_PREFIX` and `M8FLOW_KEYCLOAK_ADMIN_PASSWORD`). ECS expects the API under `/api/v1.0/*`. -The backend uses the Keycloak **superadmin** user by default (username `superadmin`, created by the Keycloak image entrypoint). Set `KEYCLOAK_ADMIN_PASSWORD` or `M8FLOW_KEYCLOAK_ADMIN_PASSWORD` to the superadmin password (same as `KEYCLOAK_SUPERADMIN_PASSWORD` when using the Keycloak image entrypoint) so the backend can create realms and run partial import. +The backend uses the Keycloak **super-admin** user by default (username `super-admin`, created by the Keycloak image entrypoint). Set `KEYCLOAK_ADMIN_PASSWORD` or `M8FLOW_KEYCLOAK_ADMIN_PASSWORD` to the super-admin password (same as `KEYCLOAK_SUPERADMIN_PASSWORD` when using the Keycloak image entrypoint) so the backend can create realms and run partial import. Build the production backend image (target `prod`, `linux/amd64` for ECS): diff --git a/docker/README.md b/docker/README.md index c2cdca9b0..99dcf91fc 100644 --- a/docker/README.md +++ b/docker/README.md @@ -2,6 +2,8 @@ This directory contains the Docker setup for running M8Flow: Compose files, Dockerfiles for app services, and the Keycloak reverse-proxy config. +**Environment variables:** Full meanings and examples live in [docs/env-reference.md](../docs/env-reference.md). This README only adds Docker Compose–specific behavior; do not duplicate the env reference here. + --- ## File reference @@ -100,6 +102,8 @@ Use with: `docker compose -f docker/m8flow-docker-compose.yml -f docker/m8flow-d Set production values in `.env` (e.g. `KEYCLOAK_HOSTNAME`, `M8FLOW_BACKEND_DATABASE_URI`, secrets) before running. +**Docker Compose caveat:** The `m8flow-backend` service sets `KEYCLOAK_URL` and `M8FLOW_KEYCLOAK_URL` to `http://keycloak-proxy:7002` so server-side calls use the proxy, while browsers use the public URL (often `http://localhost:7002` or `http://:7002`). For all other env semantics, see [docs/env-reference.md](../docs/env-reference.md). + --- ## Non-root and ports diff --git a/docs/env-reference.md b/docs/env-reference.md new file mode 100644 index 000000000..13b366c81 --- /dev/null +++ b/docs/env-reference.md @@ -0,0 +1,29 @@ +# Environment variable reference + +This file is the **canonical** place for environment variable meanings and examples. The root [README.md](../README.md) and [docker/README.md](../docker/README.md) link here instead of repeating full definitions, to reduce drift. + +## Keycloak URLs + +- `KEYCLOAK_HOSTNAME`: Browser/public base URL used to reach Keycloak (for example `http://localhost:7002`). If clients access from another machine, use `http://:7002` (or your real hostname and port). +- `KEYCLOAK_HOSTNAME_URL`: Public Keycloak base URL Keycloak uses for token issuer (`iss`). In this repo’s Docker Compose, `KC_HOSTNAME_URL` is wired from `KEYCLOAK_HOSTNAME`; set `KEYCLOAK_HOSTNAME` consistently with how users reach Keycloak. +- `KEYCLOAK_HOSTNAME_HOST` (optional): Hostname segment passed to Keycloak as `KC_HOSTNAME` in [docker/m8flow-docker-compose.yml](../docker/m8flow-docker-compose.yml) (default `localhost`). Adjust if your deployment needs a different hostname for Keycloak’s own hostname configuration. +- `KEYCLOAK_URL` / `M8FLOW_KEYCLOAK_URL`: Backend URL for Keycloak Admin/API calls. **Docker Compose:** set by compose to `http://keycloak-proxy:7002` for `m8flow-backend` (internal network). **Local dev:** often `http://localhost:7002` to match the proxy port on the host. +- `M8FLOW_APP_PUBLIC_BASE_URL` (optional): Set when the app and Keycloak are exposed on different public hosts. If unset, `KEYCLOAK_HOSTNAME` is used for generated app-facing URLs where applicable. + +## Connector attachment paths + +For SMTP and Slack connectors: + +- `*_ATTACHMENTS_DIR`: Host/source path where files are read from. +- `*_ATTACHMENTS_USER_ACCESS_DIR`: User-visible mounted path used in service-task file selection. + +Examples: + +- `M8FLOW_CONNECTOR_SMTP_ATTACHMENTS_DIR=../email_attachments` +- `M8FLOW_CONNECTOR_SMTP_ATTACHMENTS_USER_ACCESS_DIR=/email_attachments` +- `M8FLOW_CONNECTOR_SLACK_ATTACHMENTS_DIR=../slack_attachments` +- `M8FLOW_CONNECTOR_SLACK_ATTACHMENTS_USER_ACCESS_DIR=/slack_attachments` + +## Advanced Keycloak auth configs + +For `SPIFFWORKFLOW_BACKEND_AUTH_CONFIGS` patterns (master realm, `admin-cli`, role mapping), see [extensions/m8flow-backend/keycloak/KEYCLOAK_SETUP.md](../extensions/m8flow-backend/keycloak/KEYCLOAK_SETUP.md). diff --git a/extensions/m8flow-backend/bin/run_m8flow_backend.sh b/extensions/m8flow-backend/bin/run_m8flow_backend.sh index 019b852a4..054953a62 100644 --- a/extensions/m8flow-backend/bin/run_m8flow_backend.sh +++ b/extensions/m8flow-backend/bin/run_m8flow_backend.sh @@ -38,7 +38,6 @@ fi export SPIFFWORKFLOW_BACKEND_DATABASE_URI="${M8FLOW_BACKEND_DATABASE_URI}" export SPIFFWORKFLOW_BACKEND_BPMN_SPEC_ABSOLUTE_DIR="${M8FLOW_BACKEND_BPMN_SPEC_ABSOLUTE_DIR}" -export SPIFFWORKFLOW_BACKEND_RUN_DATA_SETUP="${SPIFFWORKFLOW_BACKEND_RUN_DATA_SETUP:-false}" if [[ "${M8FLOW_BACKEND_UPGRADE_DB:-true}" != "false" ]]; then cd "$repo_root/spiffworkflow-backend" diff --git a/extensions/m8flow-backend/src/m8flow_backend/api.yml b/extensions/m8flow-backend/src/m8flow_backend/api.yml index b908703d2..1b96a5d24 100644 --- a/extensions/m8flow-backend/src/m8flow_backend/api.yml +++ b/extensions/m8flow-backend/src/m8flow_backend/api.yml @@ -3,7 +3,7 @@ info: version: 1.0.0 title: m8flow-backend-extension paths: - /health: + /ping: get: summary: Health check for load balancers and monitoring description: Returns 200 when the process is up. Public, no authentication required. diff --git a/extensions/m8flow-backend/src/m8flow_backend/config.py b/extensions/m8flow-backend/src/m8flow_backend/config.py index 90cfe95d8..3f1162ec0 100644 --- a/extensions/m8flow-backend/src/m8flow_backend/config.py +++ b/extensions/m8flow-backend/src/m8flow_backend/config.py @@ -30,8 +30,8 @@ def keycloak_public_issuer_base() -> str: def keycloak_admin_user() -> str: - """Master realm admin username (default: super-admin, created by Keycloak entrypoint).""" - return _get("KEYCLOAK_ADMIN_USER") or _get("M8FLOW_KEYCLOAK_ADMIN_USER") or "super-admin" + """Master realm admin username (default is created by Keycloak entrypoint).""" + return _get("KEYCLOAK_ADMIN_USER") or _get("M8FLOW_KEYCLOAK_ADMIN_USER") def keycloak_admin_password() -> str: diff --git a/extensions/m8flow-backend/src/m8flow_backend/tenancy.py b/extensions/m8flow-backend/src/m8flow_backend/tenancy.py index c149b2a22..7e8fea58a 100644 --- a/extensions/m8flow-backend/src/m8flow_backend/tenancy.py +++ b/extensions/m8flow-backend/src/m8flow_backend/tenancy.py @@ -28,7 +28,7 @@ TENANT_CONTEXT_EXEMPT_PATH_PREFIXES: tuple[str, ...] = ( "/.well-known", "/favicon.ico", - "/v1.0/health", + "/v1.0/ping", "/v1.0/healthy", "/v1.0/status", "/v1.0/openapi.json", diff --git a/extensions/m8flow-backend/src/m8flow_backend/utils/openapi_merge.py b/extensions/m8flow-backend/src/m8flow_backend/utils/openapi_merge.py index 5fccea33f..823170c3e 100644 --- a/extensions/m8flow-backend/src/m8flow_backend/utils/openapi_merge.py +++ b/extensions/m8flow-backend/src/m8flow_backend/utils/openapi_merge.py @@ -35,7 +35,7 @@ def unified_add_api(self, specification, **kwargs): # 3. Define extension path prefix and root-level paths (no prefix) ext_prefix = "/m8flow" - ROOT_LEVEL_EXTENSION_PATHS = ("/health",) + ROOT_LEVEL_EXTENSION_PATHS = ("/ping",) # 4. Merge Paths - keep core paths unchanged, add extension paths with prefix # Core paths remain as-is; root-level extension paths stay as-is, others get /m8flow prefix diff --git a/sample.env b/sample.env index 73dc696c1..5850113a7 100644 --- a/sample.env +++ b/sample.env @@ -39,9 +39,9 @@ M8FLOW_BACKEND_BPMN_SPEC_ABSOLUTE_DIR=./process_models M8FLOW_TEMPLATES_STORAGE_DIR=./templates # --- Auth (OIDC / Keycloak) --- -# KEYCLOAK_HOSTNAME=http://localhost:7002 # URL used to reach Keycloak; set to http://:7002 when accessing from another machine -# KEYCLOAK_HOSTNAME_URL=http://localhost:7002 # Public Keycloak base URL; tokens use this as issuer (iss) -# With Docker Compose, KEYCLOAK_URL is set to http://keycloak-proxy:7002 so the backend uses the proxy. +# KEYCLOAK_HOSTNAME=http://localhost:7002 +# KEYCLOAK_HOSTNAME_URL=http://localhost:7002 +# See docs: README Keycloak Setup (hostname/issuer notes). M8FLOW_KEYCLOAK_URL=http://localhost:7002 M8FLOW_BACKEND_OPEN_ID_SERVER_URL=http://localhost:7002/realms/spiffworkflow M8FLOW_BACKEND_OPEN_ID_CLIENT_ID=spiffworkflow-backend @@ -54,10 +54,8 @@ M8FLOW_KEYCLOAK_SPOKE_KEYSTORE_P12=extensions/m8flow-backend/keystore.p12 # Keystore password; required for tenant login flow when using spoke keystore. M8FLOW_KEYCLOAK_SPOKE_KEYSTORE_PASSWORD= -# Optional: SPIFFWORKFLOW_BACKEND_AUTH_CONFIGS for Keycloak (identifier should match frontend cookie). -# For master-realm super-admin browser access, add a second config: identifier=master, uri=.../realms/master. -# If you also use API-style admin-cli tokens, add additional_valid_client_ids=admin-cli; the backend can map -# matching M8Flow realm roles from realm_access.roles when no groups claim is present. +# Optional Keycloak auth configs (identifier should match frontend cookie). +# Advanced master/admin-cli config notes: extensions/m8flow-backend/keycloak/KEYCLOAK_SETUP.md SPIFFWORKFLOW_BACKEND_AUTH_CONFIGS__0__identifier=spiffworkflow-local SPIFFWORKFLOW_BACKEND_AUTH_CONFIGS__0__label=Keycloak SPIFFWORKFLOW_BACKEND_AUTH_CONFIGS__0__uri=http://localhost:7002/realms/spiffworkflow-local @@ -69,7 +67,7 @@ SPIFFWORKFLOW_BACKEND_AUTH_CONFIGS__1__client_id=spiffworkflow-backend SPIFFWORKFLOW_BACKEND_AUTH_CONFIGS__1__client_secret=JXeQExm0JhQPLumgHtIIqf52bDalHz0q SPIFFWORKFLOW_BACKEND_AUTH_CONFIGS__1__additional_valid_client_ids=admin-cli SPIFFWORKFLOW_BACKEND_OPEN_ID_ADDITIONAL_VALID_ISSUERS=http://localhost:7002/realms/master,http://localhost:7002/realms/m8flow -# Optional local-development bootstrap for the master realm browser client and global admin user. +# Optional local bootstrap for master realm browser client and global admin user. # M8FLOW_KEYCLOAK_MASTER_CLIENT_SECRET=JXeQExm0JhQPLumgHtIIqf52bDalHz0q # KEYCLOAK_SUPER_ADMIN_USER=super-admin # KEYCLOAK_SUPER_ADMIN_PASSWORD=super-admin @@ -78,7 +76,7 @@ SPIFFWORKFLOW_BACKEND_OPEN_ID_ADDITIONAL_VALID_ISSUERS=http://localhost:7002/rea M8FLOW_BACKEND_CELERY_ENABLED=true M8FLOW_BACKEND_CELERY_BROKER_URL=redis://localhost:6379/0 M8FLOW_BACKEND_CELERY_RESULT_BACKEND=redis://localhost:6379/0 -# Backend/API should be false; the worker/flower launchers override this inside their own containers. +# Keep false for backend/API; worker and flower launchers override. # M8FLOW_BACKEND_RUNNING_IN_CELERY_WORKER=false M8FLOW_BACKEND_RUN_BACKGROUND_SCHEDULER_IN_CREATE_APP=true M8FLOW_BACKEND_CELERY_ENABLE_EVENTS=true @@ -90,34 +88,34 @@ M8FLOW_BACKEND_CELERY_AUTOSCALE_MAX=4 # M8FLOW_BACKEND_CELERY_CONCURRENCY=2 # ---Flower --- -# Optional basic auth for Flower, format user:password (empty disables auth). +# Optional Flower basic auth: user:password (empty disables). M8FLOW_BACKEND_CELERY_FLOWER_BASIC_AUTH=admin:admin # Optional worker/flower mount overrides. # M8FLOW_BACKEND_CELERY_PROCESS_MODELS_MOUNT_SOURCE=process_models_cache # --- Connector proxy --- -# CONNECTOR_PROXY_PORT=8004 -# M8FLOW_BACKEND_CONNECTOR_PROXY_URL=http://localhost:8004 +CONNECTOR_PROXY_PORT=8004 +M8FLOW_BACKEND_CONNECTOR_PROXY_URL=http://localhost:8004 # --- M8Flow Connector: SMTP --- -# Local directory for connector to map attachments to. +# SMTP attachment source dir (host path). M8FLOW_CONNECTOR_SMTP_ATTACHMENTS_DIR=../email_attachments -# Mapped volume name for attachments directory. Users will select files from this directory when attaching files in the service task. E.g.: "/email_attachments/file.pdf". +# SMTP attachment user-visible mount path in service tasks. M8FLOW_CONNECTOR_SMTP_ATTACHMENTS_USER_ACCESS_DIR=/email_attachments M8FLOW_CONNECTOR_SMTP_ATTACHMENTS_LIMIT_IN_MB=10 M8FLOW_CONNECTOR_SMTP_TIMEOUT_SECONDS=300 # --- M8Flow Connector: Slack --- -# Local directory for connector to map Slack attachments from (host path for sync). +# Slack attachment source dir (host path). M8FLOW_CONNECTOR_SLACK_ATTACHMENTS_DIR=../slack_attachments -# Mapped volume path for attachments directory. Users select files from this path when attaching in the service task. E.g.: "/slack_attachments/sample.txt". +# Slack attachment user-visible mount path in service tasks. M8FLOW_CONNECTOR_SLACK_ATTACHMENTS_USER_ACCESS_DIR=/slack_attachments M8FLOW_CONNECTOR_SLACK_UPLOAD_FILE_LIMIT_MB=50 # --- Permissions --- M8FLOW_BACKEND_PERMISSIONS_FILE_ABSOLUTE_PATH=./extensions/m8flow-backend/src/m8flow_backend/config/permissions/m8flow.yml #M8FLOW_BACKEND_PERMISSIONS_FILE_NAME=m8flow.yml -# Multitenant: true = tenant selection at / and /tenants/check; false = single-tenant. +# Multitenant mode: true enables tenant selection routes. MULTI_TENANT_ON=false # --- SQLAlchemy Logs --- @@ -128,27 +126,27 @@ M8FLOW_BACKEND_ENCRYPTION_LIB=cryptography M8FLOW_BACKEND_ENCRYPTION_KEY=0123456789abcdef0123456789abcdef # --- Tenancy --- -# Default tenant id; one Keycloak realm per tenant. +# Default tenant id. M8FLOW_DEFAULT_TENANT_ID=default M8FLOW_ALLOW_MISSING_TENANT_CONTEXT=true -# JWT claim name for tenant id. Default: m8flow_tenant_id +# JWT claim name for tenant id. # M8FLOW_TENANT_CLAIM=m8flow_tenant_id # --- Keycloak --- KEYCLOAK_ADMIN=admin -# Keycloak admin app: URL, realm, admin user. Backend accepts KEYCLOAK_URL (or M8FLOW_KEYCLOAK_URL). -# Set KEYCLOAK_HOSTNAME to the URL users/browser use to reach Keycloak so the backend accepts the token iss claim (e.g. http://localhost:7002 or http://localhost:7002). Required when using Docker or a proxy. +# Keycloak admin app settings. +# See docs: README Keycloak Setup and docs/env-reference.md. KEYCLOAK_HOSTNAME=http://localhost:7002 -# When Keycloak and the app are on different hosts, set M8FLOW_APP_PUBLIC_BASE_URL (e.g. https://app.example.com or http://localhost:7001). When not set, KEYCLOAK_HOSTNAME is used. +# Optional public app URL when app and Keycloak are on different hosts. # M8FLOW_APP_PUBLIC_BASE_URL= KEYCLOAK_URL=http://localhost:7002 KEYCLOAK_REALM=tenant-a KEYCLOAK_ADMIN_USER=admin -# Admin password for create-realm API (KEYCLOAK_ADMIN_PASSWORD or M8FLOW_KEYCLOAK_ADMIN_PASSWORD). +# Admin password for create-realm API. KEYCLOAK_ADMIN_PASSWORD=admin KEYCLOAK_CLIENT_ID=my-backend-app KEYCLOAK_CLIENT_SECRET= -# Optional mTLS: KEYCLOAK_CERT_FILE, KEYCLOAK_KEY_FILE. +# Optional mTLS cert/key paths. # KEYCLOAK_CERT_FILE= # KEYCLOAK_KEY_FILE= # Keycloak DB (used by docker/m8flow-docker-compose.yml for keycloak-db and keycloak service)