Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/self-managed/concepts/databases/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ For an architectural explanation of how secondary storage fits into Camunda 8, s
| Web Modeler | Yes | No |
| Management Identity | Yes | No |

Use this matrix as a quick decision aid when planning your deployment topology.
Use this matrix as a compatibility summary for the main Camunda components and their supported database backends.

## Document-store backends

Expand All @@ -37,7 +37,7 @@ Related documentation:

## Relational databases (RDBMS)

Camunda also supports several relational databases for secondary storage, enabling Operate, Tasklist, Identity, and REST APIs to run without Elasticsearch or OpenSearch.
Camunda also supports several relational databases for secondary storage, enabling the Orchestration Cluster API, Operate, Tasklist, and Admin to run without Elasticsearch or OpenSearch.

RDBMS and document-store backends are both valid secondary storage options. Select based on your workload, operational model, and platform standards.

Expand Down
14 changes: 13 additions & 1 deletion docs/self-managed/concepts/databases/relational-db/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Relational database (RDBMS) support is a core capability of the Orchestration Cl

RDBMS is a standard secondary storage option for Orchestration Cluster installations, alongside Elasticsearch and OpenSearch.

With RDBMS secondary storage, data is exported to your relational database, and the Orchestration Cluster reads from it.
With RDBMS secondary storage, the RDBMS exporter writes orchestration data to your relational database, and the Orchestration Cluster API queries it. Operate, Tasklist, and Admin use that API rather than querying the database directly.

In this context:

Expand All @@ -18,6 +18,18 @@ In this context:

For a deeper conceptual comparison, see [understanding primary vs secondary storage](/self-managed/concepts/secondary-storage/no-secondary-storage.md#about-this-mode).

```mermaid
graph LR
exporter["RDBMS Exporter"]
api["Orchestration Cluster API"]
rdbms["RDBMS\n(secondary storage)"]
exporter -->|"Write"| rdbms
api -->|"Query"| rdbms
style exporter fill:#e4eef8,stroke:#2272c9,color:#14082c
style api fill:#e4eef8,stroke:#2272c9,color:#14082c
style rdbms fill:#fde8da,stroke:#fc5d0d,color:#14082c
```

## Start here

- New to RDBMS in Camunda: [End-to-end RDBMS setup guide](/self-managed/concepts/databases/relational-db/rdbms-setup-guide.md)
Expand Down
49 changes: 38 additions & 11 deletions docs/self-managed/concepts/secondary-storage/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@ title: "Secondary storage"
description: "Learn how secondary storage works in Camunda Self-Managed environments, how it interacts with the Zeebe engine, and how to configure or manage it effectively."
---

import DataFlowImg from '../img/diagram-secondary-storage-data-flow.png';

Camunda uses a layered storage model that separates workflow execution data from data used by web applications and APIs.

## About secondary storage
Expand All @@ -27,7 +25,7 @@ Camunda supports multiple secondary storage backends.
For the latest list of supported database versions, see the
[RDBMS version support policy](/self-managed/concepts/databases/relational-db/rdbms-support-policy.md).

Both document-store and RDBMS backends are valid secondary storage choices in Self-Managed deployments. Support maturity can vary by product area and version (for example, Orchestration Cluster APIs, Operate, Tasklist, Identity, or Optimize), so confirm current compatibility details before choosing a backend.
Both document-store and RDBMS backends are valid secondary storage choices in Self-Managed deployments. Support maturity can vary by product area and version (for example, the Orchestration Cluster API, Operate, Tasklist, Admin, or Optimize), so confirm current compatibility details before choosing a backend.

| Database type | Availability | Use case |
| :--------------------- | :------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
Expand All @@ -38,21 +36,50 @@ Both document-store and RDBMS backends are valid secondary storage choices in Se
Camunda 8 supports both [Amazon OpenSearch](https://aws.amazon.com/opensearch-service) and the open-source [OpenSearch](https://opensearch.org/) distribution.
:::

```mermaid
graph TD
subgraph oc["Orchestration Cluster"]
broker["Zeebe Broker"]
exp_rdbms["RDBMS Exporter"]
exp_camunda["Camunda Exporter\n(for Orchestration Cluster on ES/OS)"]
exp_opt["Elasticsearch/OpenSearch Exporter\n(for Optimize)"]
api["Orchestration Cluster API"]
apps["Operate · Tasklist · Admin"]
end
rdbms["RDBMS\n(PostgreSQL, Oracle, MariaDB, MySQL)"]
es["Elasticsearch/OpenSearch"]
opt["Optimize\n(document-store required)"]
broker -->|"Enable for RDBMS backend"| exp_rdbms
broker -.->|"Enable for document-store backend"| exp_camunda
broker -.->|"Enable when Optimize is deployed"| exp_opt
exp_rdbms -->|"Write"| rdbms
exp_camunda -->|"Write"| es
exp_opt -.->|"Write"| es
apps -->|"Use"| api
api -->|"Query selected backend"| rdbms
api -.->|"Query selected backend"| es
opt -.->|"Read/Write (if deployed)"| es
style broker fill:#e4eef8,stroke:#2272c9,color:#14082c
style exp_rdbms fill:#e4eef8,stroke:#2272c9,color:#14082c
style exp_camunda fill:#e4eef8,stroke:#2272c9,color:#14082c
style exp_opt fill:#e4eef8,stroke:#2272c9,color:#14082c
style api fill:#e4eef8,stroke:#2272c9,color:#14082c
style apps fill:#e4eef8,stroke:#2272c9,color:#14082c
style rdbms fill:#fde8da,stroke:#fc5d0d,color:#14082c
style es fill:#fde8da,stroke:#fc5d0d,color:#14082c
style oc fill:#f0f5ff,stroke:#2272c9
style opt fill:#e8fdf1,stroke:#10c95d,color:#14082c
```

:::note
Starting in 8.9, Camunda 8 Run and default lightweight installs use H2 as the default secondary storage. Elasticsearch remains a supported alternative in Camunda 8 Run. OpenSearch and RDBMS-based secondary storage are supported in Self-Managed deployments. Enable the backend you need explicitly when required.

H2 is a convenience default for local development, testing, and evaluation. It is not a production reference architecture and is not a valid backend for multi-broker Helm clusters.
:::

### Data flow

The following diagram shows how secondary storage fits into the Camunda data flow.

<img src={DataFlowImg} alt="Camunda data flow showing secondary storage" class="img-noborder img-transparent"/>

1. The Zeebe broker executes workflow instances and stores state in primary storage.
1. The exporter, part of Zeebe, streams workflow and task data to secondary storage.
1. Orchestration Cluster applications and APIs (for example, Operate, Tasklist, Identity, and the Orchestration Cluster REST API) read data from secondary storage.
1. Exporters, running as part of Zeebe, write orchestration data to the configured secondary storage backend and can write to multiple targets when needed.
1. Operate, Tasklist, and Admin use the Orchestration Cluster API, which reads data from the configured secondary storage backend.

## Choosing a secondary storage backend

Expand Down
6 changes: 6 additions & 0 deletions docs/self-managed/deployment/helm/install/helm-with-rdbms.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,12 @@ In Camunda 8, secondary storage stores historical data and process state. You ca

When using RDBMS, **Optimize still requires Elasticsearch or OpenSearch**. Only the Orchestration Cluster uses RDBMS.

In this topology:

- The RDBMS exporter writes Orchestration Cluster data to your external relational database.
- Operate, Tasklist, and Admin use the Orchestration Cluster API, and that API queries the configured RDBMS secondary storage.
- If you also deploy Optimize, keep Elasticsearch or OpenSearch available and enable an additional Elasticsearch/OpenSearch exporter for Optimize.

## Prerequisites

Before you begin:
Expand Down
38 changes: 25 additions & 13 deletions docs/self-managed/deployment/manual/rdbms/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,37 +13,49 @@ Manual installation is **not** supported for Kubernetes. If you run on Kubernete

## Secondary storage architecture

The Orchestration Cluster reads from a single configured secondary storage type (RDBMS in this deployment). However, the Zeebe broker can export to multiple targets simultaneously. If you deploy Optimize, configure both the RDBMS exporter (for Orchestration Cluster operations) and a document-store exporter (Elasticsearch/OpenSearch) for Optimize.
The Orchestration Cluster reads from a single configured secondary storage type (RDBMS in this deployment). However, the Zeebe broker can export to multiple targets simultaneously. If you deploy Optimize, configure both the RDBMS exporter (for Orchestration Cluster operations) and an Elasticsearch/OpenSearch exporter for Optimize.

```mermaid
graph LR
subgraph oc[Orchestration Cluster]
broker[Broker]
broker[Zeebe Broker]
rdbms_exporter[RDBMS Exporter]
doc_exporter[Elasticsearch/OpenSearch Exporter\n(for Optimize)]
api[Orchestration Cluster API]
apps[Operate · Tasklist · Admin]
end

subgraph storage[Secondary Storage]
rdbms[RDBMS]
es[Elasticsearch/OpenSearch]
subgraph storage[Data Stores]
rdbms[RDBMS secondary storage]
es[Elasticsearch/OpenSearch for Optimize]
end

opt[Optimize]

broker -->|Export| rdbms
broker -.->|Export| es
api -->|Read| rdbms
broker -->|Export orchestration data| rdbms_exporter
rdbms_exporter -->|Write| rdbms
apps -->|Use| api
broker -.->|Optional export for Optimize| doc_exporter
doc_exporter -.->|Optional write| es
api -->|Query| rdbms
opt -->|Read/Write| es

style rdbms fill:#e1f5ff
style es fill:#fff3e0
style opt fill:#f3e5f5
style rdbms_exporter fill:#e4eef8,stroke:#2272c9,color:#14082c
style doc_exporter fill:#e4eef8,stroke:#2272c9,color:#14082c
style api fill:#e4eef8,stroke:#2272c9,color:#14082c
style apps fill:#e4eef8,stroke:#2272c9,color:#14082c
style rdbms fill:#fde8da,stroke:#fc5d0d,color:#14082c
style es fill:#fde8da,stroke:#fc5d0d,color:#14082c
style opt fill:#e8fdf1,stroke:#10c95d,color:#14082c
style oc fill:#f0f5ff,stroke:#2272c9
style storage fill:#fff8f4,stroke:#fc5d0d
```

**Key points:**

- Requests to the Orchestration Cluster API read from the configured secondary storage (RDBMS).
- Operate, Tasklist, and Admin use the Orchestration Cluster API, and that API reads from the configured secondary storage (RDBMS).
- Optimize requires Elasticsearch or OpenSearch and reads and writes directly to it.
- The Orchestration Cluster exports to multiple targets simultaneously to support this architecture.
- The Zeebe broker can export to multiple targets simultaneously to support this architecture.

## Supported installation targets

Expand Down
Loading