diff --git a/docs/self-managed/concepts/databases/overview.md b/docs/self-managed/concepts/databases/overview.md index 7cb37e27199..230410b45fd 100644 --- a/docs/self-managed/concepts/databases/overview.md +++ b/docs/self-managed/concepts/databases/overview.md @@ -20,7 +20,7 @@ For an architectural explanation of how secondary storage fits into Camunda 8, s | Web Modeler | Yes | No | | Management Identity | Yes | No | -Use this matrix as a quick decision aid when planning your deployment topology. +Use this matrix as a compatibility summary for the main Camunda components and their supported database backends. ## Document-store backends @@ -37,7 +37,7 @@ Related documentation: ## Relational databases (RDBMS) -Camunda also supports several relational databases for secondary storage, enabling Operate, Tasklist, Identity, and REST APIs to run without Elasticsearch or OpenSearch. +Camunda also supports several relational databases for secondary storage, enabling the Orchestration Cluster API, Operate, Tasklist, and Admin to run without Elasticsearch or OpenSearch. RDBMS and document-store backends are both valid secondary storage options. Select based on your workload, operational model, and platform standards. diff --git a/docs/self-managed/concepts/databases/relational-db/index.md b/docs/self-managed/concepts/databases/relational-db/index.md index b76e33a744f..eaf060f1fd0 100644 --- a/docs/self-managed/concepts/databases/relational-db/index.md +++ b/docs/self-managed/concepts/databases/relational-db/index.md @@ -9,7 +9,7 @@ Relational database (RDBMS) support is a core capability of the Orchestration Cl RDBMS is a standard secondary storage option for Orchestration Cluster installations, alongside Elasticsearch and OpenSearch. -With RDBMS secondary storage, data is exported to your relational database, and the Orchestration Cluster reads from it. +With RDBMS secondary storage, the RDBMS exporter writes orchestration data to your relational database, and the Orchestration Cluster API queries it. Operate, Tasklist, and Admin use that API rather than querying the database directly. In this context: @@ -18,6 +18,18 @@ In this context: For a deeper conceptual comparison, see [understanding primary vs secondary storage](/self-managed/concepts/secondary-storage/no-secondary-storage.md#about-this-mode). +```mermaid +graph LR + exporter["RDBMS Exporter"] + api["Orchestration Cluster API"] + rdbms["RDBMS\n(secondary storage)"] + exporter -->|"Write"| rdbms + api -->|"Query"| rdbms + style exporter fill:#e4eef8,stroke:#2272c9,color:#14082c + style api fill:#e4eef8,stroke:#2272c9,color:#14082c + style rdbms fill:#fde8da,stroke:#fc5d0d,color:#14082c +``` + ## Start here - New to RDBMS in Camunda: [End-to-end RDBMS setup guide](/self-managed/concepts/databases/relational-db/rdbms-setup-guide.md) diff --git a/docs/self-managed/concepts/secondary-storage/index.md b/docs/self-managed/concepts/secondary-storage/index.md index e0005054d38..2d64e57cd3a 100644 --- a/docs/self-managed/concepts/secondary-storage/index.md +++ b/docs/self-managed/concepts/secondary-storage/index.md @@ -4,8 +4,6 @@ title: "Secondary storage" description: "Learn how secondary storage works in Camunda Self-Managed environments, how it interacts with the Zeebe engine, and how to configure or manage it effectively." --- -import DataFlowImg from '../img/diagram-secondary-storage-data-flow.png'; - Camunda uses a layered storage model that separates workflow execution data from data used by web applications and APIs. ## About secondary storage @@ -27,7 +25,7 @@ Camunda supports multiple secondary storage backends. For the latest list of supported database versions, see the [RDBMS version support policy](/self-managed/concepts/databases/relational-db/rdbms-support-policy.md). -Both document-store and RDBMS backends are valid secondary storage choices in Self-Managed deployments. Support maturity can vary by product area and version (for example, Orchestration Cluster APIs, Operate, Tasklist, Identity, or Optimize), so confirm current compatibility details before choosing a backend. +Both document-store and RDBMS backends are valid secondary storage choices in Self-Managed deployments. Support maturity can vary by product area and version (for example, the Orchestration Cluster API, Operate, Tasklist, Admin, or Optimize), so confirm current compatibility details before choosing a backend. | Database type | Availability | Use case | | :--------------------- | :------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | @@ -38,21 +36,50 @@ Both document-store and RDBMS backends are valid secondary storage choices in Se Camunda 8 supports both [Amazon OpenSearch](https://aws.amazon.com/opensearch-service) and the open-source [OpenSearch](https://opensearch.org/) distribution. ::: +```mermaid +graph TD + subgraph oc["Orchestration Cluster"] + broker["Zeebe Broker"] + exp_rdbms["RDBMS Exporter"] + exp_camunda["Camunda Exporter\n(for Orchestration Cluster on ES/OS)"] + exp_opt["Elasticsearch/OpenSearch Exporter\n(for Optimize)"] + api["Orchestration Cluster API"] + apps["Operate · Tasklist · Admin"] + end + rdbms["RDBMS\n(PostgreSQL, Oracle, MariaDB, MySQL)"] + es["Elasticsearch/OpenSearch"] + opt["Optimize\n(document-store required)"] + broker -->|"Enable for RDBMS backend"| exp_rdbms + broker -.->|"Enable for document-store backend"| exp_camunda + broker -.->|"Enable when Optimize is deployed"| exp_opt + exp_rdbms -->|"Write"| rdbms + exp_camunda -->|"Write"| es + exp_opt -.->|"Write"| es + apps -->|"Use"| api + api -->|"Query selected backend"| rdbms + api -.->|"Query selected backend"| es + opt -.->|"Read/Write (if deployed)"| es + style broker fill:#e4eef8,stroke:#2272c9,color:#14082c + style exp_rdbms fill:#e4eef8,stroke:#2272c9,color:#14082c + style exp_camunda fill:#e4eef8,stroke:#2272c9,color:#14082c + style exp_opt fill:#e4eef8,stroke:#2272c9,color:#14082c + style api fill:#e4eef8,stroke:#2272c9,color:#14082c + style apps fill:#e4eef8,stroke:#2272c9,color:#14082c + style rdbms fill:#fde8da,stroke:#fc5d0d,color:#14082c + style es fill:#fde8da,stroke:#fc5d0d,color:#14082c + style oc fill:#f0f5ff,stroke:#2272c9 + style opt fill:#e8fdf1,stroke:#10c95d,color:#14082c +``` + :::note Starting in 8.9, Camunda 8 Run and default lightweight installs use H2 as the default secondary storage. Elasticsearch remains a supported alternative in Camunda 8 Run. OpenSearch and RDBMS-based secondary storage are supported in Self-Managed deployments. Enable the backend you need explicitly when required. H2 is a convenience default for local development, testing, and evaluation. It is not a production reference architecture and is not a valid backend for multi-broker Helm clusters. ::: -### Data flow - -The following diagram shows how secondary storage fits into the Camunda data flow. - -Camunda data flow showing secondary storage - 1. The Zeebe broker executes workflow instances and stores state in primary storage. -1. The exporter, part of Zeebe, streams workflow and task data to secondary storage. -1. Orchestration Cluster applications and APIs (for example, Operate, Tasklist, Identity, and the Orchestration Cluster REST API) read data from secondary storage. +1. Exporters, running as part of Zeebe, write orchestration data to the configured secondary storage backend and can write to multiple targets when needed. +1. Operate, Tasklist, and Admin use the Orchestration Cluster API, which reads data from the configured secondary storage backend. ## Choosing a secondary storage backend diff --git a/docs/self-managed/deployment/helm/install/helm-with-rdbms.md b/docs/self-managed/deployment/helm/install/helm-with-rdbms.md index 38fa09d171e..c4b1d285523 100644 --- a/docs/self-managed/deployment/helm/install/helm-with-rdbms.md +++ b/docs/self-managed/deployment/helm/install/helm-with-rdbms.md @@ -30,6 +30,12 @@ In Camunda 8, secondary storage stores historical data and process state. You ca When using RDBMS, **Optimize still requires Elasticsearch or OpenSearch**. Only the Orchestration Cluster uses RDBMS. +In this topology: + +- The RDBMS exporter writes Orchestration Cluster data to your external relational database. +- Operate, Tasklist, and Admin use the Orchestration Cluster API, and that API queries the configured RDBMS secondary storage. +- If you also deploy Optimize, keep Elasticsearch or OpenSearch available and enable an additional Elasticsearch/OpenSearch exporter for Optimize. + ## Prerequisites Before you begin: diff --git a/docs/self-managed/deployment/manual/rdbms/index.md b/docs/self-managed/deployment/manual/rdbms/index.md index ca8cbe6a93b..5be3187847a 100644 --- a/docs/self-managed/deployment/manual/rdbms/index.md +++ b/docs/self-managed/deployment/manual/rdbms/index.md @@ -13,37 +13,49 @@ Manual installation is **not** supported for Kubernetes. If you run on Kubernete ## Secondary storage architecture -The Orchestration Cluster reads from a single configured secondary storage type (RDBMS in this deployment). However, the Zeebe broker can export to multiple targets simultaneously. If you deploy Optimize, configure both the RDBMS exporter (for Orchestration Cluster operations) and a document-store exporter (Elasticsearch/OpenSearch) for Optimize. +The Orchestration Cluster reads from a single configured secondary storage type (RDBMS in this deployment). However, the Zeebe broker can export to multiple targets simultaneously. If you deploy Optimize, configure both the RDBMS exporter (for Orchestration Cluster operations) and an Elasticsearch/OpenSearch exporter for Optimize. ```mermaid graph LR subgraph oc[Orchestration Cluster] - broker[Broker] + broker[Zeebe Broker] + rdbms_exporter[RDBMS Exporter] + doc_exporter["Elasticsearch/OpenSearch Exporter\n(for Optimize)"] api[Orchestration Cluster API] + apps[Operate · Tasklist · Admin] end - subgraph storage[Secondary Storage] - rdbms[RDBMS] - es[Elasticsearch/OpenSearch] + subgraph storage[Data Stores] + rdbms[RDBMS secondary storage] + es["Elasticsearch/OpenSearch for Optimize"] end opt[Optimize] - broker -->|Export| rdbms - broker -.->|Export| es - api -->|Read| rdbms + broker -->|Export orchestration data| rdbms_exporter + rdbms_exporter -->|Write| rdbms + apps -->|Use| api + broker -.->|Optional export for Optimize| doc_exporter + doc_exporter -.->|Optional write| es + api -->|Query| rdbms opt -->|Read/Write| es - style rdbms fill:#e1f5ff - style es fill:#fff3e0 - style opt fill:#f3e5f5 + style rdbms_exporter fill:#e4eef8,stroke:#2272c9,color:#14082c + style doc_exporter fill:#e4eef8,stroke:#2272c9,color:#14082c + style api fill:#e4eef8,stroke:#2272c9,color:#14082c + style apps fill:#e4eef8,stroke:#2272c9,color:#14082c + style rdbms fill:#fde8da,stroke:#fc5d0d,color:#14082c + style es fill:#fde8da,stroke:#fc5d0d,color:#14082c + style opt fill:#e8fdf1,stroke:#10c95d,color:#14082c + style oc fill:#f0f5ff,stroke:#2272c9 + style storage fill:#fff8f4,stroke:#fc5d0d ``` **Key points:** -- Requests to the Orchestration Cluster API read from the configured secondary storage (RDBMS). +- Operate, Tasklist, and Admin use the Orchestration Cluster API, and that API reads from the configured secondary storage (RDBMS). - Optimize requires Elasticsearch or OpenSearch and reads and writes directly to it. -- The Orchestration Cluster exports to multiple targets simultaneously to support this architecture. +- The Zeebe broker can export to multiple targets simultaneously to support this architecture. ## Supported installation targets