Skip to content

Commit d894b50

Browse files
kbatuigaskbatuigas
committed
Add Cloud API commands for shadowing and failover
1 parent 3aa7217 commit d894b50

File tree

5 files changed

+395
-24
lines changed

5 files changed

+395
-24
lines changed

‎local-antora-playbook.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ content:
1717
- url: https://github.com/redpanda-data/docs
1818
branches: [v/*, shared, site-search,'!v-end-of-life/*']
1919
- url: https://github.com/redpanda-data/cloud-docs
20-
branches: 'main'
20+
branches: 'DOC-1621-Document-Cloud-Feature-Shadowing-Disaster-Recovery-Enterprise'
2121
- url: https://github.com/redpanda-data/redpanda-labs
2222
branches: main
2323
start_paths: [docs,'*/docs']

‎modules/manage/pages/disaster-recovery/shadowing/failover-runbook.adoc‎

Lines changed: 122 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,11 @@ rpk cluster info --brokers shadow-cluster-1.example.com:9092,shadow-cluster-2.ex
7373

7474
Check the health of your shadow links:
7575

76+
[tabs]
77+
======
78+
rpk::
79+
+
80+
--
7681
[,bash]
7782
----
7883
# List all shadow links
@@ -86,6 +91,27 @@ rpk shadow status <shadow-link-name>
8691
----
8792
8893
For detailed command options, see xref:reference:rpk/rpk-shadow/rpk-shadow-list.adoc[`rpk shadow list`], xref:reference:rpk/rpk-shadow/rpk-shadow-describe.adoc[`rpk shadow describe`], and xref:reference:rpk/rpk-shadow/rpk-shadow-status.adoc[`rpk shadow status`].
94+
--
95+
96+
ifdef::env-cloud[]
97+
Cloud API::
98+
+
99+
--
100+
[,bash]
101+
----
102+
# List all shadow links
103+
curl 'https://api.redpanda.com/v1/shadow-links' \
104+
-H "Content-Type: application/json" \
105+
-H "Authorization: Bearer ${RP_CLOUD_TOKEN}"
106+
107+
# Check the configuration of your shadow link
108+
curl https://api.redpanda.com/v1/shadow-links/<shadow-link-id> \
109+
-H "Content-Type: application/json" \
110+
-H "Authorization: Bearer ${RP_CLOUD_TOKEN}"
111+
----
112+
--
113+
endif::[]
114+
======
89115

90116
Verify that the following conditions exist before proceeding with failover:
91117

@@ -152,22 +178,76 @@ Note the replication lag to estimate potential data loss during failover. The `T
152178

153179
A complete cluster failover is appropriate If you observe that the source cluster is no longer reachable:
154180

181+
[tabs]
182+
======
183+
rpk::
184+
+
185+
--
155186
[,bash]
156187
----
157188
# Fail over all topics in the shadow link
158189
rpk shadow failover <shadow-link-name> --all
159190
----
191+
--
192+
193+
ifdef::env-cloud[]
194+
Cloud API::
195+
+
196+
--
197+
[,bash]
198+
----
199+
# Get Data Plane API URL of shadow cluster
200+
export DATAPLANE_API_URL=`curl https://api.cloud.redpanda.com/v1/clusters/<shadow-cluster-id> \
201+
-H "Content-Type: application/json" \
202+
-H "Authorization: Bearer ${RP_CLOUD_TOKEN}" | jq .cluster.dataplane_api`
203+
204+
# Fail over all topics in the shadow link
205+
curl -X POST "$DATAPLANE_API_URL/v1/shadowlink/<shadow-link-name>/failover" \
206+
-H "Content-Type: application/json" \
207+
-H "Authorization: Bearer ${RP_CLOUD_TOKEN}" \
208+
-d '{
209+
"name": "<shadow-link-name>"
210+
}'
211+
----
212+
--
213+
endif::[]
214+
======
160215

161216
For detailed command options, see xref:reference:rpk/rpk-shadow/rpk-shadow-failover.adoc[`rpk shadow failover`].
162217

163218
For selective topic failover (when only specific services are affected):
164219

220+
[tabs]
221+
======
222+
rpk::
223+
+
224+
--
225+
[,bash]
226+
----
227+
# Fail over individual topics
228+
rpk shadow failover <shadow-link-name> --topic <topic-name-1>
229+
rpk shadow failover <shadow-link-name> --topic <topic-name-2>
230+
----
231+
--
232+
233+
ifdef::env-cloud[]
234+
Cloud API::
235+
+
236+
--
165237
[,bash]
166238
----
167239
# Fail over individual topics
168-
rpk shadow failover <shadow-link-name> --topic <topic-name>
169-
rpk shadow failover <shadow-link-name> --topic <topic-name>
240+
curl -X POST "$DATAPLANE_API_URL/v1/shadowlink/<shadow-link-name>/failover" \
241+
-H "Content-Type: application/json" \
242+
-H "Authorization: Bearer ${RP_CLOUD_TOKEN}" \
243+
-d '{
244+
"name": "<shadow-link-name>",
245+
"shadowTopicName": "<topic-name-1>"
246+
}'
170247
----
248+
--
249+
endif::[]
250+
======
171251

172252
[[monitor-progress]]
173253
=== Monitor failover progress
@@ -232,13 +312,33 @@ Test message production and consumption, consumer group functionality, and criti
232312

233313
After all applications are running normally:
234314

315+
[tabs]
316+
======
317+
rpk::
318+
+
319+
--
235320
[,bash]
236321
----
237322
# Optional: Delete the shadow link (no longer needed)
238323
rpk shadow delete <shadow-link-name>
239324
----
240325
241326
For detailed command options, see xref:reference:rpk/rpk-shadow/rpk-shadow-delete.adoc[`rpk shadow delete`].
327+
--
328+
329+
ifdef::env-cloud[]
330+
Cloud API::
331+
+
332+
--
333+
# Optional: Delete the shadow link (no longer needed)
334+
curl -X DELETE https://api.redpanda.com/v1/shadow-links/<shadow-link-id> \
335+
-H "Content-Type: application/json" \
336+
-H "Authorization: Bearer ${RP_CLOUD_TOKEN}"
337+
--
338+
339+
For the full API reference, see link:/api/doc/cloud-controlplane/v1/operation/operation-[Control Plane API reference].
340+
endif::[]
341+
======
242342

243343
Document the time of failover initiation and completion, applications affected and recovery times, data loss estimates based on replication lag, and issues encountered during failover.
244344

@@ -252,11 +352,31 @@ Document the time of failover initiation and completion, applications affected a
252352

253353
If topics remain stuck after addressing these cluster health issues and you need immediate failover, you can force delete the shadow link to failover all topics:
254354

355+
[tabs]
356+
======
357+
rpk::
358+
+
359+
--
255360
[,bash]
256361
----
257362
# Force delete the shadow link to failover all topics
258363
rpk shadow delete <shadow-link-name> --force
259364
----
365+
--
366+
367+
ifdef::env-cloud[]
368+
Cloud API::
369+
+
370+
--
371+
curl -X DELETE https://api.redpanda.com/v1/shadow-links/<shadow-link-id> \
372+
-H "Content-Type: application/json" \
373+
-H "Authorization: Bearer ${RP_CLOUD_TOKEN}" \
374+
-d '{
375+
"force": true
376+
}'
377+
--
378+
endif::[]
379+
======
260380

261381
[WARNING]
262382
====

‎modules/manage/pages/disaster-recovery/shadowing/failover.adoc‎

Lines changed: 89 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ endif::[]
1414
Failover is the process of modifying shadow topics or an entire shadow cluster from read-only replicas to fully writable resources, and ceasing replication from the source cluster. You can fail over individual topics for selective workload migration or fail over the entire cluster for comprehensive disaster recovery. This critical operation transforms your shadow resources into operational production assets, allowing you to redirect application traffic when the source cluster becomes unavailable.
1515

1616
ifdef::env-cloud[]
17-
You can failover a shadow link using the Redpanda Cloud UI, `rpk`, or the Data Plane API.
17+
You can failover a shadow link using the Redpanda Cloud UI, `rpk`, or the Data Plane API.
1818
endif::[]
1919

2020
ifndef::env-cloud[]
@@ -41,36 +41,124 @@ NOTE: To avoid a split-brain scenario after failover, ensure that all clients ar
4141

4242
== Failover commands
4343

44+
ifdef::env-cloud[]
45+
=== Get Data Plane API URL
46+
47+
If using the Data Plane API, run the following to get the Data Plane API URL of the shadow cluster:
48+
49+
[,bash]
50+
----
51+
export DATAPLANE_API_URL=`curl https://api.cloud.redpanda.com/v1/clusters/<shadow-cluster-id> \
52+
-H "Content-Type: application/json" \
53+
-H "Authorization: Bearer ${RP_CLOUD_TOKEN}" | jq .cluster.dataplane_api`
54+
----
55+
endif::[]
56+
4457
You can perform failover at different levels of granularity to match your disaster recovery needs:
4558

4659
=== Individual topic failover
4760

4861
To fail over a specific shadow topic while leaving other topics in the shadow link still replicating, run:
4962

63+
[tabs]
64+
======
65+
rpk::
66+
+
67+
--
5068
[,bash]
5169
----
5270
rpk shadow failover <shadow-link-name> --topic <topic-name>
5371
----
5472
5573
Use this approach when you need to selectively failover specific workloads or when testing failover procedures. For detailed command options, see xref:reference:rpk/rpk-shadow/rpk-shadow-failover.adoc[`rpk shadow failover`].
74+
--
75+
76+
ifdef::env-cloud[]
77+
Data Plane API::
78+
+
79+
--
80+
Send a `POST /v1/shadowlink/\{shadow_link_name}/failover` request to the Data Plane API. Specify the name of the shadow topic in the request body:
81+
82+
[,bash]
83+
----
84+
curl -X POST "$DATAPLANE_API_URL/v1/shadowlink/<shadow-link-name>/failover" \
85+
-H "Content-Type: application/json" \
86+
-H "Authorization: Bearer ${RP_CLOUD_TOKEN}" \
87+
-d '{
88+
"name": "<shadow-link-name>",
89+
"shadowTopicName": "<shadow-topic-name>"
90+
}'
91+
----
92+
--
93+
endif::[]
94+
======
5695

5796
=== Complete shadow link failover (cluster failover)
5897

5998
To fail over all shadow topics associated with the shadow link simultaneously, run:
6099

100+
[tabs]
101+
======
102+
rpk::
103+
+
104+
--
61105
[,bash]
62106
----
63107
rpk shadow failover <shadow-link-name> --all
64108
----
109+
--
110+
111+
ifdef::env-cloud[]
112+
Data Plane API::
113+
+
114+
--
115+
Send a `POST /v1/shadowlink/\{shadow_link_name}/failover` request to the Data Plane API. If you do not specify a shadow topic in the request body, this command requests a failover of all shadow topics associated with the shadow link:
116+
[,bash]
117+
----
118+
curl -X POST "$DATAPLANE_API_URL/v1/shadowlink/<shadow-link-name>/failover" \
119+
-H "Content-Type: application/json" \
120+
-H "Authorization: Bearer ${RP_CLOUD_TOKEN}" \
121+
-d '{
122+
"name": "<shadow-link-name>"
123+
}'
124+
----
125+
--
126+
endif::[]
127+
======
65128

66129
Use this approach during a complete regional disaster when you need to activate the entire shadow cluster as your new production environment.
67130

68131
=== Force delete shadow link (emergency failover)
69132

133+
[tabs]
134+
======
135+
rpk::
136+
+
137+
--
70138
[,bash]
71139
----
72140
rpk shadow delete <shadow-link-name> --force
73141
----
142+
--
143+
144+
ifdef::env-cloud[]
145+
Control Plane API::
146+
+
147+
--
148+
Use the Control Plane API to force delete a shadow link:
149+
150+
[,bash]
151+
----
152+
curl -X DELETE 'https://api.redpanda.com/v1/shadow-links/<shadow-link-id>' \
153+
-H "Content-Type: application/json" \
154+
-H "Authorization: Bearer ${RP_CLOUD_TOKEN}" \
155+
-d '{
156+
"force": true
157+
}'
158+
----
159+
--
160+
endif::[]
161+
======
74162

75163
[WARNING]
76164
====

‎modules/manage/pages/disaster-recovery/shadowing/monitor.adoc‎

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,19 +19,59 @@ include::shared:partial$emergency-shadowing-callout.adoc[]
1919

2020
To list existing shadow links, run:
2121

22+
[tabs]
23+
======
24+
rpk::
25+
+
26+
--
2227
[,bash]
2328
----
2429
rpk shadow list
2530
----
31+
--
32+
33+
ifdef::env-cloud[]
34+
Control Plane API::
35+
+
36+
--
37+
[,bash]
38+
----
39+
curl 'https://api.redpanda.com/v1/shadow-links' \
40+
-H "Content-Type: application/json" \
41+
-H "Authorization: Bearer ${RP_CLOUD_TOKEN}"
42+
----
43+
--
44+
endif::[]
45+
======
2646

2747
To view shadow link configuration details, run:
2848

49+
[tabs]
50+
======
51+
rpk::
52+
+
53+
--
2954
[,bash]
3055
----
31-
rpk shadow describe <my-disaster-recovery-link>
56+
rpk shadow describe <shadow-link-name>
3257
----
3358
3459
For detailed command options, see xref:reference:rpk/rpk-shadow/rpk-shadow-list.adoc[`rpk shadow list`] and xref:reference:rpk/rpk-shadow/rpk-shadow-describe.adoc[`rpk shadow describe`]. This command shows the complete configuration of the shadow link, including connection settings, filters, and synchronization options.
60+
--
61+
62+
ifdef::env-cloud[]
63+
Control Plane API::
64+
+
65+
--
66+
[,bash]
67+
----
68+
curl 'https://api.redpanda.com/v1/shadow-links/<shadow-link-id>' \
69+
-H "Content-Type: application/json" \
70+
-H "Authorization: Bearer ${RP_CLOUD_TOKEN}"
71+
----
72+
--
73+
endif::[]
74+
======
3575

3676
To check your shadow link status and ensure proper operation, run:
3777

0 commit comments

Comments
 (0)