Skip to content

Commit c97ff62

Browse files
Bueller87Bueller87
authored
Add Elasticsearch advanced visibility example with Kafka
Adds values.postgres-es7.yaml example demonstrating Cadence deployment with: - PostgreSQL for primary persistence - Elasticsearch 7 for advanced visibility - Kafka for async visibility processing with pre-provisioned topics Key configuration: - Disables SQL visibility store when Elasticsearch is enabled - Pre-provisions Kafka topics (__consumer_offsets, cadence-visibility, cadence-visibility-dlq) with replicationFactor: 1 - Uses visibilityIndex: "cadence-visibility-es7" matching ES template pattern - Explicitly disables Kafka in postgres-only example (values.postgres.yaml) Also fixes schema-server-job.yaml to skip SQL visibility database creation when Elasticsearch is enabled.
1 parent 88ca9b6 commit c97ff62

File tree

5 files changed

+226
-5
lines changed

5 files changed

+226
-5
lines changed

‎charts/cadence/Chart.lock‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,5 +8,11 @@ dependencies:
88
- name: mysql
99
repository: oci://registry-1.docker.io/bitnamicharts
1010
version: 12.3.5
11-
digest: sha256:9d2bdf5e27b9fedc3a32d7abcf6c3ff569818e2756240f94977b1047efafe4b0
12-
generated: "2025-09-17T16:04:57.7251556+02:00"
11+
- name: elasticsearch
12+
repository: oci://registry-1.docker.io/bitnamicharts
13+
version: 21.6.3
14+
- name: kafka
15+
repository: oci://registry-1.docker.io/bitnamicharts
16+
version: 30.1.8
17+
digest: sha256:794d39e9345acfb117730c963e7818fa388613b2cbb2becef8c3ce43d1e15206
18+
generated: "2025-11-03T14:00:44.205655-08:00"

‎charts/cadence/Chart.yaml‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
apiVersion: v2
22
name: cadence
3-
version: 1.0.10
3+
version: 1.1.0
44
appVersion: v1.3.6
55

66
description: |
@@ -40,3 +40,11 @@ dependencies:
4040
version: 12.x.x
4141
repository: oci://registry-1.docker.io/bitnamicharts
4242
condition: mysql.enabled
43+
- name: elasticsearch
44+
version: 21.6.x
45+
repository: oci://registry-1.docker.io/bitnamicharts
46+
condition: elasticsearch.enabled
47+
- name: kafka
48+
version: 30.x.x
49+
repository: oci://registry-1.docker.io/bitnamicharts
50+
condition: kafka.enabled

‎charts/cadence/README.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# cadence
22

3-
![Version: 1.0.10](https://img.shields.io/badge/Version-1.0.10-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v1.3.6](https://img.shields.io/badge/AppVersion-v1.3.6-informational?style=flat-square)
3+
![Version: 1.1.0](https://img.shields.io/badge/Version-1.1.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v1.3.6](https://img.shields.io/badge/AppVersion-v1.3.6-informational?style=flat-square)
44

55
Cadence is a distributed, scalable, durable, and highly available orchestration engine
66
to execute asynchronous long-running business logic in a scalable and resilient way.
@@ -24,6 +24,8 @@ This chart deploys Uber Cadence server components and web UI.
2424
| Repository | Name | Version |
2525
|------------|------|---------|
2626
| oci://registry-1.docker.io/bitnamicharts | cassandra | 11.x.x |
27+
| oci://registry-1.docker.io/bitnamicharts | elasticsearch | 21.6.x |
28+
| oci://registry-1.docker.io/bitnamicharts | kafka | 30.x.x |
2729
| oci://registry-1.docker.io/bitnamicharts | mysql | 12.x.x |
2830
| oci://registry-1.docker.io/bitnamicharts | postgresql | 16.x.x |
2931

‎charts/cadence/examples/values.postgres-es7.yaml‎

Lines changed: 203 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,203 @@
1+
# Namespace note: install into namespace: cadence-postgres-es7
2+
3+
# Allow Bitnami charts to use legacy repository images
4+
global:
5+
image:
6+
tag: "v1.3.6" # Default version
7+
security:
8+
allowInsecureImages: true
9+
10+
# Force Cadence to use PostgreSQL for main DB
11+
config:
12+
persistence:
13+
# Name of the default datastore (PostgreSQL)
14+
defaultStore: "default"
15+
# Disable basic SQL visibility (we're using ES-only via Kafka)
16+
visibilityStore: ""
17+
# Name of the advanced visibility datastore (Elasticsearch)
18+
# Note: This defines the datastore, but dynamic config controls how it's used
19+
advancedVisibilityStore: "es-visibility"
20+
database:
21+
driver: "postgres"
22+
sql:
23+
hosts: "cadence-release-postgresql.cadence-postgres-es7.svc.cluster.local"
24+
port: 5432
25+
dbname: "cadence"
26+
user: "cadence"
27+
password: "changeme-strong"
28+
tls:
29+
enabled: false
30+
sslMode: ""
31+
elasticsearch:
32+
enabled: true
33+
version: "v7"
34+
user: ""
35+
password: ""
36+
protocol: "http"
37+
hosts: "cadence-release-elasticsearch.cadence-postgres-es7.svc.cluster.local"
38+
port: 9200
39+
visibilityIndex: "cadence-visibility-es7"
40+
tls:
41+
enabled: false
42+
kafka: # needed by elasticsearch integration
43+
enabled: true
44+
brokers: "cadence-release-kafka.cadence-postgres-es7.svc.cluster.local"
45+
46+
# Enable Cadence schema jobs
47+
schema:
48+
serverJob:
49+
enabled: true
50+
elasticSearchJob:
51+
enabled: true
52+
53+
# Ensure Cadence uses Elasticsearch for advanced visibility
54+
# Note: "es" is a special keyword for advanced visibility, not the datastore name
55+
dynamicConfig:
56+
values:
57+
system.writeVisibilityStoreName:
58+
- value: "es"
59+
system.readVisibilityStoreName:
60+
- value: "es"
61+
62+
### subcharts values which can be omited if user has their own deployment
63+
64+
# Deploy Postgres within the same release (Bitnami subchart)
65+
postgresql:
66+
enabled: true
67+
image:
68+
registry: docker.io
69+
repository: bitnamilegacy/postgresql
70+
tag: "16.4.0"
71+
pullPolicy: IfNotPresent
72+
auth:
73+
username: cadence
74+
password: "changeme-strong"
75+
database: cadence
76+
primary:
77+
persistence:
78+
enabled: true
79+
size: 8Gi
80+
81+
# Deploy Elasticsearch with single node mode
82+
elasticsearch:
83+
enabled: true
84+
master:
85+
masterOnly: false
86+
replicaCount: 1
87+
resources:
88+
requests:
89+
cpu: 1
90+
memory: 1024Mi
91+
limits:
92+
cpu: 2
93+
memory: 2048Mi
94+
data:
95+
replicaCount: 0
96+
coordinating:
97+
replicaCount: 0
98+
ingest:
99+
replicaCount: 0
100+
sysctlImage:
101+
enabled: false
102+
image:
103+
tag: 7.17.23
104+
repository: bitnamilegacy/elasticsearch
105+
106+
kafka:
107+
enabled: true
108+
image:
109+
registry: docker.io
110+
repository: bitnamilegacy/kafka
111+
tag: "3.8.0"
112+
pullPolicy: IfNotPresent
113+
# KRaft mode configuration (Kafka without ZooKeeper)
114+
kraft:
115+
enabled: true
116+
# Disable ZooKeeper since we're using KRaft
117+
zookeeper:
118+
enabled: false
119+
# Controller and broker configuration for KRaft
120+
controller:
121+
replicaCount: 1
122+
persistence:
123+
enabled: true
124+
size: 8Gi
125+
resources:
126+
requests:
127+
cpu: 500m
128+
memory: 1Gi
129+
limits:
130+
cpu: 1
131+
memory: 2Gi
132+
# GKE Autopilot compatibility
133+
podSecurityContext:
134+
fsGroup: 1001
135+
runAsUser: 1001
136+
containerSecurityContext:
137+
runAsNonRoot: true
138+
allowPrivilegeEscalation: false
139+
broker:
140+
replicaCount: 1
141+
persistence:
142+
enabled: true
143+
size: 8Gi
144+
resources:
145+
requests:
146+
cpu: 500m
147+
memory: 1Gi
148+
limits:
149+
cpu: 1
150+
memory: 2Gi
151+
# GKE Autopilot compatibility
152+
podSecurityContext:
153+
fsGroup: 1001
154+
runAsUser: 1001
155+
containerSecurityContext:
156+
runAsNonRoot: true
157+
allowPrivilegeEscalation: false
158+
# GKE Autopilot compatibility: disable privileged init container
159+
sysctlImage:
160+
enabled: false
161+
# Kafka topic configuration for Cadence visibility
162+
autoCreateTopicsEnable: true
163+
numPartitions: 4
164+
defaultReplicationFactor: 1
165+
# Configure replication factors for internal topics (demo/single-node setup)
166+
offsetsTopicReplicationFactor: 1
167+
transactionStateLogReplicationFactor: 1
168+
transactionStateLogMinIsr: 1
169+
# Provision topics at startup to avoid race conditions
170+
provisioning:
171+
enabled: true
172+
topics:
173+
- name: __consumer_offsets
174+
partitions: 50
175+
replicationFactor: 1
176+
config:
177+
cleanup.policy: compact
178+
compression.type: producer
179+
- name: cadence-visibility
180+
partitions: 4
181+
replicationFactor: 1
182+
- name: cadence-visibility-dlq
183+
partitions: 4
184+
replicationFactor: 1
185+
# Listener configuration
186+
listeners:
187+
client:
188+
protocol: PLAINTEXT
189+
controller:
190+
protocol: PLAINTEXT
191+
interbroker:
192+
protocol: PLAINTEXT
193+
# Service configuration
194+
service:
195+
type: ClusterIP
196+
ports:
197+
client: 9092
198+
199+
# Do NOT deploy Cassandra or MySQL
200+
cassandra:
201+
enabled: false
202+
mysql:
203+
enabled: false

‎charts/cadence/examples/values.postgres.yaml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,9 +41,11 @@ postgresql:
4141
enabled: true
4242
size: 8Gi
4343

44-
# Do NOT deploy ES, Cassandra or MySQL
44+
# Do NOT deploy ES, Kafka, Cassandra or MySQL
4545
elasticsearch:
4646
enabled: false
47+
kafka:
48+
enabled: false
4749
cassandra:
4850
enabled: false
4951
mysql:

0 commit comments

Comments
 (0)