Skip to content

Commit 59aabf4

Browse files
micheleRPDeflaimunFeediver1JakeSCahillpgellert
authored
Doc 773 single source leader pinning (#873)
Co-authored-by: Paulo Borges <[email protected]> Co-authored-by: Joyce Fee <[email protected]> Co-authored-by: Jake Cahill <[email protected]> Co-authored-by: Gellért Peresztegi-Nagy <[email protected]> Co-authored-by: Angela Simms <[email protected]> Co-authored-by: Kat Batuigas <[email protected]> Co-authored-by: JakeSCahill <[email protected]>
1 parent 3559a16 commit 59aabf4

File tree

2 files changed

+37
-16
lines changed

2 files changed

+37
-16
lines changed

modules/develop/pages/consume-data/follower-fetching.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,9 +20,9 @@ The first fetch from a consumer is processed by a Redpanda leader broker. The le
2020

2121
Redpanda decides which replica a consumer fetches from. If the consumer configures its `client.rack` property, Redpanda by default selects a replica from the same rack as the consumer, if available.
2222

23+
ifndef::env-cloud[]
2324
To enable follower fetching in Redpanda, configure properties for the consumer and the Redpanda cluster and broker:
2425

25-
ifndef::env-cloud[]
2626
- For a Redpanda cluster, set the xref:reference:cluster-properties.adoc#enable_rack_awareness[`enable_rack_awareness`] property to `true`.
2727
- For each Redpanda broker, set the xref:reference:node-properties.adoc#rack[`rack`] property to a rack ID.
2828
- For each consumer, set the `client.rack` property to a rack ID.
Lines changed: 36 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,45 +1,64 @@
11
= Leader Pinning
22
// tag::single-source[]
3-
:description: Learn about Leader Pinning and how to configure a preferred partition leader location based on cloud availability zones or regions.
3+
:description: Learn about leader pinning and how to configure a preferred partition leader location based on cloud availability zones or regions.
4+
5+
Produce requests that write data to Redpanda topics go through the topic partition leader, which syncs messages across its follower replicas. For a Redpanda cluster deployed across multiple availability zones (AZs), leader pinning ensures that a topic's partition leaders are geographically closer to clients, which helps decrease networking costs and guarantees lower latency.
6+
7+
If consumers are located in the same preferred region or AZ for leader pinning, and you have not set up xref:develop:consume-data/follower-fetching.adoc[follower fetching], leader pinning can also help reduce networking costs on consume requests.
8+
9+
ifndef::env-cloud[]
10+
== Prerequisites
411

512
[NOTE]
613
====
714
include::shared:partial$enterprise-license.adoc[]
815
====
916

10-
Produce requests that write data to Redpanda topics go through the topic partition leader, which syncs messages across its follower replicas. For a Redpanda cluster deployed across multiple availability zones (AZs), Leader Pinning ensures that a topic's partition leaders are geographically closer to clients, which helps decrease networking costs and guarantees lower latency.
17+
Before you can enable leader pinning, you must xref:manage:rack-awareness.adoc#configure-rack-awareness[configure rack awareness] on the cluster. If the config_ref:enable_rack_awareness,true,properties/cluster-properties[] cluster configuration property is set to `false`, leader pinning is disabled across the cluster.
1118

12-
If consumers are located in the same preferred region or AZ for Leader Pinning, and you have not set up xref:develop:consume-data/follower-fetching.adoc[follower fetching], Leader Pinning can also help reduce networking costs on consume requests.
19+
endif::[]
1320

14-
== Prerequisites
21+
ifndef::env-cloud[]
22+
== Configure leader pinning
1523

16-
Before you can enable Leader Pinning, you must xref:manage:rack-awareness.adoc#configure-rack-awareness[configure rack awareness] on the cluster. If the config_ref:enable_rack_awareness,true,properties/cluster-properties[] cluster configuration property is set to `false`, Leader Pinning is disabled across the cluster.
17-
18-
== Configure Leader Pinning
19-
20-
You can use both a topic configuration property and a cluster configuration property to configure Leader Pinning.
21-
22-
You can set the topic configuration property for individual topics only, or set the cluster-wide configuration property that will enable Leader Pinning by default for all topics. You can also use a combination in which a default setting applies across the cluster, and you toggle the setting on or off for specific topics.
24+
You can use both a topic configuration property and a cluster configuration property to configure leader pinning.
2325

26+
You can set the topic configuration property for individual topics only, or set the cluster-wide configuration property that will enable leader pinning by default for all topics. You can also use a combination in which a default setting applies across the cluster, and you toggle the setting on or off for specific topics.
2427

2528
This configuration is based on the following scenario: you have Redpanda deployed in a multi-AZ or multi-region cluster, and you have configured each broker so that the config_ref:rack,true,properties/broker-properties[] configuration property contains rack IDs corresponding to the AZ IDs:
2629

2730
* Set the topic configuration property xref:reference:properties/topic-properties.adoc#redpandaleaderspreference[`redpanda.leaders.preference`]. The property accepts the following string values:
2831
+
2932
--
30-
** `none`: Opt out the topic from Leader Pinning.
33+
** `none`: Opt out the topic from leader pinning.
3134
** `racks:<rack1>[,<rack2>,...]`: Specify the preferred location (rack) of all topic partition leaders. The list can contain one or more rack IDs, and you can list the IDs in any order. Spaces in the list are ignored, for example: `racks:rack1,rack2` and `racks: rack1, rack2` are equivalent. You cannot specify empty rack IDs, for example: `racks: rack1,,rack2`. If you specify multiple IDs, Redpanda tries to distribute the partition leader locations equally across brokers in these racks.
3235
--
3336
+
3437
This property inherits the default value from the cluster property `default_leaders_preference`.
3538
36-
* Set the cluster configuration property config_ref:default_leaders_preference,true,properties/cluster-properties[], which specifies the default Leader Pinning configuration for all topics that don’t have `redpanda.leaders.preference` explicitly set. It accepts values in the same format as `redpanda.leaders.preference`. Default: `none`
39+
* Set the cluster configuration property config_ref:default_leaders_preference,true,properties/cluster-properties[], which specifies the default leader pinning configuration for all topics that don’t have `redpanda.leaders.preference` explicitly set. It accepts values in the same format as `redpanda.leaders.preference`. Default: `none`
3740
+
3841
This property also affects internal topics, such as `__consumer_offsets` and transaction coordinators. All offset tracking and transaction coordination requests get placed within the preferred regions or AZs for all clients, so you see end-to-end latency and networking cost benefits.
3942
40-
If there is more than one broker in the preferred AZ (or AZs), Leader Pinning distributes partition leaders uniformly across brokers in the AZ.
43+
If there is more than one broker in the preferred AZ (or AZs), leader pinning distributes partition leaders uniformly across brokers in the AZ.
44+
45+
endif::[]
4146

42-
== Leader Pinning failover across availability zones
47+
ifdef::env-cloud[]
48+
== Configure leader pinning
49+
50+
Configure leader pinning if you have Redpanda deployed in a multi-AZ or multi-region cluster and your ingress is concentrated in a particular AZ or region.
51+
52+
Use the topic configuration property `redpanda.leaders.preference` to configure leader pinning for individual topics. The property accepts the following string values:
53+
54+
** `none`: Opt out the topic from leader pinning.
55+
** `racks:<rack1>[,<rack2>,...]`: Specify the preferred location (rack) of all topic partition leaders. The list can contain one or more rack IDs, and you can list the IDs in any order. Spaces in the list are ignored, for example: `racks:rack1,rack2` and `racks: rack1, rack2` are equivalent. You cannot specify empty rack IDs, for example: `racks: rack1,,rack2`. If you specify multiple IDs, Redpanda tries to distribute the partition leader locations equally across brokers in these racks.
56+
57+
If there is more than one broker in the preferred AZ (or AZs), leader pinning distributes partition leaders uniformly across brokers in the AZ.
58+
59+
endif::[]
60+
61+
== Leader pinning failover across availability zones
4362

4463
If there are three AZs: A, B, and C, and A becomes unavailable, the failover behavior is as follows:
4564

@@ -50,3 +69,5 @@ If there are three AZs: A, B, and C, and A becomes unavailable, the failover beh
5069
== Suggested reading
5170

5271
* xref:develop:consume-data/follower-fetching.adoc[]
72+
73+
// end::single-source[]

0 commit comments

Comments
 (0)