Skip to content

Clarify support for cross region Tiered Storage / RRR #942

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kbatuigas merged 5 commits into from
Jan 14, 2025
Merged

Clarify support for cross region Tiered Storage / RRR #942

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
49f7b56
Cross-region is supported
kbatuigas Jan 8, 2025
13c5d93
Explicitly state multi-region buckets are not currently supported
kbatuigas Jan 9, 2025
867ef06
Clarify cross region support per provider
kbatuigas Jan 13, 2025
25ea723
Update modules/manage/partials/remote-read-replicas.adoc
kbatuigas Jan 13, 2025
e145dfe
Apply suggestions from review
kbatuigas Jan 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 3 additions & 9 deletions modules/manage/partials/remote-read-replicas.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,6 @@ You can create Remote Read Replica topics in a Redpanda cluster that directly ac
- When upgrading to Redpanda 23.2, metadata from object storage is not synchronized until all brokers in the cluster are upgraded. If you need to force a mixed-version cluster to sync read replicas, move partition leadership to brokers running the original version.
====

TIP: To create a Remote Read Replica topic in another region, consider using a https://aws.amazon.com/s3/features/multi-region-access-points/[multi-region bucket^] to simplify deployment and optimize performance.

ifdef::env-kubernetes[]
helm_ref:storage.tiered.config[]
endif::[]
Expand All @@ -36,11 +34,9 @@ endif::[]

You need the following:

* An origin cluster with xref:{tiered-storage-link}#set-up-tiered-storage[Tiered Storage] set up.
* An origin cluster with xref:{tiered-storage-link}#set-up-tiered-storage[Tiered Storage] set up. Note that multi-region buckets or containers are not supported.
Copy link
Contributor Author

@kbatuigas kbatuigas Jan 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that it's specifically the object storage bucket (for the TS setup) that has to be single-region and not multi-region. Would like to double check that this is the best place in this doc to add a heads-up regarding multi-region?

Copy link

@Lazin Lazin Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with keeping it this way. RRR will work with both single reagion and multi-region buckets but we only test with single region.

* A topic on the origin cluster, which you can use as a Remote Read Replica topic on the remote cluster.
* A separate remote cluster in the same region as the bucket or container used for the origin cluster.
** If you use a multi-region bucket/container, you can create the read replica cluster in any region that has that bucket/container.
** If you use a single-region bucket/container, the remote cluster must be in the same region as the bucket/container.
* A separate remote cluster. This cluster can be in the same or a different region as the bucket or container used for the origin cluster.
Copy link
Contributor Author

@kbatuigas kbatuigas Jan 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our version of RRR doc for Cloud says:

Prerequisites

  • A BYOC reader cluster in Ready state. This separate reader cluster must exist in the same Redpanda organization and the same cloud provider account and region as the source cluster.

Does this need to be edited? It seems to me that the prereq regarding the region can be changed, but I'm not sure about it the reader cluster having to be in the same RP org and cloud account still.

Copy link

@Lazin Lazin Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The edit makes sense.

Copy link
Contributor

@frenchfrywpepper frenchfrywpepper Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To further complicate things, different region is only supported in GCP right now. Not in AWS. RRR is not at all supported in Azure.

Copy link
Contributor

@Deflaimun Deflaimun Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's that deep, maybe we should add a table or something about the 3 different providers.

Copy link
Contributor

@micheleRP micheleRP Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, interesting: thanks @frenchfrywpepper! I've added to this other PR about beta/LA features. Please also update that PR with any other info you know about beta/LA features.

I don't think a table for the providers is necessary here. I'd just note the difference for providers in the Prereqs.


include::shared:partial$enterprise-license.adoc[]

Expand All @@ -58,9 +54,7 @@ You must configure access to the same object storage as the origin cluster.
ifndef::env-kubernetes[]
To set up a Remote Read Replica topic on a separate remote cluster:

. Create a remote cluster for the Remote Read Replica topic.
* If that's a multi-region bucket/container, you can create the read replica cluster in any region that has that bucket/container.
* If that's a single-region bucket/container, the remote cluster must be in the same region as the bucket/container.
. Create a remote cluster for the Remote Read Replica topic. The remote cluster can be in the same or a different region as the bucket/container.
Copy link
Contributor

@micheleRP micheleRP Jan 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should you have the same info here in the step; i.e., for AWS, the remote cluster must be in the same region as the origin cluster, but for GCP, the remote cluster can be in the same or a different region as the origin cluster.

. Run `rpk cluster config edit`, and then specify properties specific to your object storage provider (your cluster will require a restart after any changes to these properties):
+
|===
Expand Down
1 change: 1 addition & 0 deletions modules/manage/partials/tiered-storage.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ rpk cluster license info

- Migrating topics from one object storage provider to another is not supported.
- Migrating topics from one bucket or container to another is not supported.
- Multi-region buckets or containers are not supported.

CAUTION: Redpanda strongly recommends that you do not re-enable Tiered Storage after previously enabling and disabling it. Re-enabling Tiered Storage can result in inconsistent data and data gaps in Tiered Storage for a topic.

Expand Down
Loading