Skip to content

Commit 0cd3c64

Browse files
asimms41asimms41
authored
DOC-767 Add secret management to RPCN in Cloud (#138)
* add secret management * update style of placeholders * address review comments * update with review comments * typo * package update for Bloblang playground
1 parent 533c84e commit 0cd3c64

File tree

3 files changed

+221
-0
lines changed

3 files changed

+221
-0
lines changed

‎modules/ROOT/nav.adoc‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,7 @@
6868
*** xref:develop:connect/configuration/error_handling.adoc[]
6969
*** xref:develop:connect/configuration/interpolation.adoc[]
7070
*** xref:develop:connect/configuration/field_paths.adoc[]
71+
*** xref:develop:connect/configuration/secret-management.adoc[Manage Secrets]
7172
*** xref:develop:connect/configuration/processing_pipelines.adoc[]
7273
*** xref:develop:connect/configuration/monitor-connect.adoc[Monitor Data Pipelines]
7374
*** xref:develop:connect/configuration/scale-pipelines.adoc[Scale Data Pipelines]

‎modules/develop/pages/connect/configuration/secret-management.adoc‎

Lines changed: 219 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,219 @@
1+
= Manage Secrets
2+
:description: Learn how to manage secrets in Redpanda Connect using the Cloud UI or Data Plane API, and how to add them to your data pipelines.
3+
4+
Learn how to manage secrets in Redpanda Connect, and how to add them to your data pipelines without exposing them.
5+
6+
Secrets are stored in the secret management solution of your Cloud provider and are retrieved when you run a pipeline configuration that references them.
7+
8+
== Prerequisites
9+
10+
* A running BYOC, Dedicated or Serverless cluster
11+
* On BYOC clusters only, check that secrets management is enabled:
12+
13+
.. Log in to https://cloud.redpanda.com[Redpanda Cloud^].
14+
.. Go to the **Connect** page.
15+
.. Select the **Redpanda Connect** tab and make sure you can see a **Secrets** tab.
16+
17+
+
18+
If you cannot see a **Secrets** tab, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda Support^]
19+
20+
== Manage secrets
21+
22+
You can manage secrets from the Cloud UI or Data Plane API.
23+
24+
=== Create a secret
25+
26+
You can create a secret and reference it in multiple data pipelines on the same cluster.
27+
28+
[tabs]
29+
=====
30+
Cloud UI (BYOC and Dedicated)::
31+
+
32+
--
33+
. Log in to https://cloud.redpanda.com[Redpanda Cloud^].
34+
. Go to the **Connect** page.
35+
. Select the **Redpanda Connect** tab and then the **Secrets** tab.
36+
. Click **Create secret**.
37+
. In **Secret name**, enter a name for the secret. You cannot rename the secret once it is created.
38+
. In **Secret value**, enter the secret you need to add.
39+
. Click **Create secret**.
40+
+
41+
The secret details are listed in the **Secrets** tab ready to <<add-a-secret-to-a-data-pipeline,add to your data pipelines>>.
42+
43+
--
44+
45+
Cloud UI (Serverless)::
46+
+
47+
--
48+
. Log in to https://cloud.redpanda.com[Redpanda Cloud^].
49+
. Go to the **Connect** page.
50+
. Select the **Secrets** tab and click **Create secret**.
51+
. In **Secret name**, enter a name for the secret. You cannot rename the secret once it is created.
52+
. In **Secret value**, enter the secret you need to add.
53+
. Click **Create secret**.
54+
+
55+
The secret details are listed in the **Secrets** tab ready to <<add-a-secret-to-a-data-pipeline,add to your data pipelines>>.
56+
57+
--
58+
59+
Data Plane API::
60+
+
61+
--
62+
You must use a Base64-encoded secret.
63+
64+
. xref:manage:api/cloud-api-quickstart.adoc#try-the-cloud-api[Authenticate and get the base URL] for the Data Plane API.
65+
. Make a request to xref:api:ROOT:cloud-api.adoc#post-/v1alpha2/secrets[`POST /v1alpha2/secrets`].
66+
+
67+
[,bash]
68+
----
69+
curl -X POST "https://<dataplane-api-url>/v1alpha2/secrets" \
70+
-H 'accept: application/json'\
71+
-H 'authorization: Bearer <token>'\
72+
-H 'content-type: application/json' \
73+
-d '{"id":"<secret-name>","scopes":["SCOPE_REDPANDA_CONNECT"],"secret_data":"<secret-value>"}'
74+
----
75+
+
76+
You must include the following values:
77+
78+
- `<dataplane-api-url>`: The base URL for the Data Plane API.
79+
- `<token>`: The API key you generated during authentication.
80+
- `<secret-name>`: The ID or name of the secret you want to add. Use only the following characters: `^[A-Z][A-Z0-9_]*$`.
81+
- `<secret-value>`: The Base64-encoded secret.
82+
- This scope: `"SCOPE_REDPANDA_CONNECT"`.
83+
84+
+
85+
The response returns the name of the secret and the scope `"SCOPE_REDPANDA_CONNECT"`.
86+
87+
You can now <<add-a-secret-to-a-data-pipeline,add the secret to your data pipeline>>.
88+
89+
--
90+
=====
91+
92+
=== Update a secret
93+
94+
You can only update the secret value, not its name.
95+
96+
NOTE: Changes to secret values do not take effect until a pipeline is restarted.
97+
98+
[tabs]
99+
=====
100+
Cloud UI (BYOC and Dedicated)::
101+
+
102+
--
103+
. Log in to https://cloud.redpanda.com[Redpanda Cloud^].
104+
. Go to the **Connect** page.
105+
. Select the **Redpanda Connect** tab and then the **Secrets** tab.
106+
. Find the secret you want to update and click the edit icon.
107+
. Enter the new secret value and click **Update Secret**.
108+
. Start and stop any pipelines that reference the secret.
109+
110+
--
111+
112+
Cloud UI (Serverless)::
113+
+
114+
--
115+
. Log in to https://cloud.redpanda.com[Redpanda Cloud^].
116+
. Go to the **Connect** page.
117+
. Select the **Secrets** tab.
118+
. Find the secret you want to update and click the edit icon.
119+
. Enter the new secret value and click **Update Secret**.
120+
. Start and stop any pipelines that reference the secret.
121+
122+
--
123+
124+
Data Plane API::
125+
+
126+
--
127+
You must use a Base64-encoded secret.
128+
129+
. xref:manage:api/cloud-api-quickstart.adoc#try-the-cloud-api[Authenticate and get the base URL] for the Data Plane API.
130+
. Make a request to xref:api:ROOT:cloud-api.adoc#put-/v1alpha2/secrets/-id-[`PUT /v1alpha2/secrets/\{id}`].
131+
+
132+
[,bash]
133+
----
134+
curl -X PUT "https://<dataplane-api-url>/v1alpha2/secrets/<secret-name>" \
135+
-H 'accept: application/json'\
136+
-H 'authorization: Bearer <token>'\
137+
-H 'content-type: application/json' \
138+
-d '{"scopes":["SCOPE_REDPANDA_CONNECT"],"secret_data":"<secret-value>"}'
139+
----
140+
+
141+
You must include the following values:
142+
143+
- `<dataplane-api-url>`: The base URL for the Data Plane API.
144+
- `<secret-name>`: The name of the secret you want to update.
145+
- `<token>`: The API key you generated during authentication.
146+
- This scope: `"SCOPE_REDPANDA_CONNECT"`.
147+
- `<secret-value>`: Your new Base64-encoded secret.
148+
149+
+
150+
The response returns the name of the secret and the scope `"SCOPE_REDPANDA_CONNECT"`.
151+
152+
--
153+
=====
154+
155+
=== Delete a secret
156+
157+
Before you delete a secret, make sure that you remove references to it from your data pipelines.
158+
159+
NOTE: Changes do not affect pipelines that are already running.
160+
161+
162+
[tabs]
163+
=====
164+
Cloud UI (BYOC and Dedicated)::
165+
+
166+
--
167+
. Log in to https://cloud.redpanda.com[Redpanda Cloud^].
168+
. Go to the **Connect** page.
169+
. Select the **Redpanda Connect** tab and then the **Secrets** tab.
170+
. Find the secret you want to remove and click the delete icon.
171+
. Confirm your deletion.
172+
173+
--
174+
175+
Cloud UI (Serverless)::
176+
+
177+
--
178+
. Log in to https://cloud.redpanda.com[Redpanda Cloud^].
179+
. Go to the **Connect** page.
180+
. Select the **Secrets** tab.
181+
. Find the secret you want to remove and click the delete icon.
182+
. Confirm your deletion.
183+
184+
--
185+
186+
Data Plane API::
187+
+
188+
--
189+
190+
. xref:manage:api/cloud-api-quickstart.adoc#try-the-cloud-api[Authenticate and get the base URL] for the Data Plane API.
191+
. Make a request to xref:api:ROOT:cloud-api.adoc#delete-/v1alpha2/secrets/-id-[`DELETE /v1alpha2/secrets/\{id}`].
192+
+
193+
[,bash]
194+
----
195+
curl -X DELETE "https://<dataplane-api-url>/v1alpha2/secrets/<secret-name>" \
196+
-H 'accept: application/json'\
197+
-H 'authorization: Bearer <token>'\
198+
----
199+
+
200+
You must include the following values:
201+
202+
- `<dataplane-api-url>`: The base URL for the Data Plane API.
203+
- `<secret-name>`: The name of the secret you want to delete.
204+
- `<token>`: The API key you generated during authentication.
205+
--
206+
=====
207+
208+
== Add a secret to a data pipeline
209+
210+
You can add a secret to any pipeline in your cluster using the notation `${secrets.SECRET_NAME}`. In the Cloud UI, you can copy the notation from the **Secrets** tab.
211+
212+
For example:
213+
214+
```yml
215+
sasl:
216+
- mechanism: SCRAM-SHA-256
217+
username: "user"
218+
password: "${secrets.PASSWORD}"
219+
```

‎modules/develop/pages/connect/connect-quickstart.adoc‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -228,6 +228,7 @@ When you've finished experimenting with your data pipeline, you can delete the p
228228

229229
* Try one of our xref:cookbooks:index.adoc[Redpanda Connect cookbooks].
230230
* Choose xref:develop:connect/components/catalog.adoc[connectors for your use case].
231+
* Learn how to xref:develop:connect/configuration/secret-management.adoc[add secrets to your pipeline].
231232
* Learn how to xref:develop:connect/configuration/monitor-connect.adoc[monitor a data pipeline on a BYOC cluster].
232233
* Learn how to xref:develop:connect/configuration/scale-pipelines.adoc[manually scale resources for a pipeline on a BYOC cluster].
233234
* Learn how to xref:redpanda-connect:guides:getting_started.adoc[configure, test, and run a data pipeline locally].

0 commit comments

Comments
 (0)