You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/event-hubs/geo-replication.md
+20-14Lines changed: 20 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,6 +2,8 @@
2
2
title: 'Azure Event Hubs geo-replication'
3
3
description: 'This article describes the Azure Event Hubs geo-replication feature'
4
4
ms.topic: article
5
+
author: axisc
6
+
ms.author: aschhabria
5
7
ms.date: 06/10/2024
6
8
ms.custom: references_regions
7
9
---
@@ -32,14 +34,14 @@ This rest of this document is focused on the Geo-replication feature. For detail
32
34
The public preview of the Geo-replication feature is supported for namespaces in Event Hubs self-serve scaling Dedicated clusters. You can use the feature with new, or existing namespaces in dedicated self-serve clusters. The following features aren't supported with Geo-replication:
33
35
- Customer Managed Keys (CMK)
34
36
- Managed Identity for Capture
35
-
-VNet features (service endpoints, or private endpoints)
37
+
-Virtual network features (service endpoints, or private endpoints)
36
38
- Large messages support (now in public preview)
37
39
- Kafka Transactions (now in public preview)
38
40
39
41
Some of the key aspects of Geo Data Replication public preview are:
40
42
- Primary-secondary replication model – Geo-replication is built on primary-secondary replication model, where at a given time there’s only one Primary namespace that serves event producers and event consumers.
41
-
- Event Hubs performs fully managed byte-to-byte replication of metadata, event data and consumer offset across secondaries with the configured consistency levels.
42
-
- Stable namespace FQDN – The FQDN does not need to change when promotion is performed.
43
+
- Event Hubs performs fully managed byte-to-byte replication of metadata, event data, and consumer offset across secondaries with the configured consistency levels.
44
+
- Stable namespace FQDN – The FQDN doesn't need to change when promotion is performed.
43
45
- Replication consistency - There are two replication consistency settings, synchronous and asynchronous.
44
46
- User-managed promotion of a secondary to being the new primary.
45
47
@@ -70,7 +72,7 @@ With asynchronous replication enabled, all messages are committed in the primary
70
72
71
73
**Synchronous replication**
72
74
73
-
When synchronous replication is enabled, published events are replicated to the secondary, which must confirm the message before it's committed in the primary. With synchronous replication, your application publishes at the rate it takes to publish, replicate, acknowledge and commit. It also means that your application is tied to the availability of both regions. If the secondary region goes down, messages can't be acknowledged or committed.
75
+
When synchronous replication is enabled, published events are replicated to the secondary, which must confirm the message before it's committed in the primary. With synchronous replication, your application publishes at the rate it takes to publish, replicate, acknowledge, and commit. It also means that your application is tied to the availability of both regions. If the secondary region goes down, messages can't be acknowledged or committed.
74
76
75
77
**Replication consistency comparison**
76
78
@@ -82,7 +84,7 @@ With synchronous replication:
82
84
Synchronous replication provides the greatest assurance that your data is safe. If you have synchronous replication, then when it's committed, then it's committed in all of the regions configured for Geo-replication. When synchronous replication is enabled though, your application availability can be reduced due to depending on the availability of both regions.
83
85
Enabling asynchronous replication doesn't impact latency very much, and service availability isn't impacted by the loss of a secondary region. Asynchronous replication doesn’t have the absolute guarantee that all regions have the data before it's committed it like synchronous replication does. You can also set the amount of time that your secondary can be out of sync before incoming traffic is throttled. The setting can be from 5 minutes to 1,440 minutes, which is one day. If you're looking to use regions with a large distance between them, then asynchronous replication is likely the best option for you.
84
86
85
-
Replication consistency configuration can change after Geo-replication configuration. You can go from synchronous to asynchronous, or from asynchronous to synchronous. If you go from synchronous to asynchronous, your latency, and application availability improves. If you go from asynchronous to synchronous, your secondary becomes configured as synchronous after lag reaches zero. If you're running with a continual lag for whatever reason, then you may need to pause your publishers in order for lag to reach zero and your mode to be able to switch to synchronous.
87
+
Replication consistency configuration can change after Geo-replication configuration. You can go from synchronous to asynchronous, or from asynchronous to synchronous. If you go from synchronous to asynchronous, your latency, and application availability improves. If you go from asynchronous to synchronous, your secondary becomes configured as synchronous after lag reaches zero. If you're running with a continual lag for whatever reason, then you might need to pause your publishers in order for lag to reach zero and your mode to be able to switch to synchronous.
86
88
87
89
The general reasons to have synchronous replication enabled are tied to the importance of the data, specific business needs, or compliance reasons. If your primary goal is application availability rather than data assurance, then asynchronous consistency is likely the better choice.
88
90
@@ -102,18 +104,19 @@ The Geo-replication feature enables customers to configure a secondary region to
102
104
Users can monitor the progress of the replication job by monitoring the replication lag metric in Application Metrics logs.
103
105
- Enable Application Metrics logs in your Event Hubs namespace following [Monitoring Azure Event Hubs - Azure Event Hubs | Microsoft Learn](./monitor-event-hubs.md).
104
106
- Once Application Metrics logs are enabled, you need to produce and consume data from namespace for a few minutes before you start to see the logs.
105
-
- To view Application Metrics logs, navigate to Monitoring section of Event Hubs and click on the ‘Logs’ blade. You can use the following query to find the replication lag (in seconds) between the primary and secondary namespaces.
106
-
```
107
-
AzureDiagnostics
108
-
| where TimeGenerated > ago(1h)
109
-
| where Category == "ApplicationMetricsLogs"
110
-
| where ActivityName_s == "ReplicationLag
111
-
```
107
+
- To view Application Metrics logs, navigate to **Monitoring** section of Event Hubs page, and select **Logs** on the left menu. You can use the following query to find the replication lag (in seconds) between the primary and secondary namespaces.
108
+
109
+
```kusto
110
+
AzureDiagnostics
111
+
| where TimeGenerated > ago(1h)
112
+
| where Category == "ApplicationMetricsLogs"
113
+
| where ActivityName_s == "ReplicationLag
114
+
```
112
115
- The column count_d indicates the replication lag in seconds between the primary and secondary region.
113
116
114
117
## Publishing Data
115
118
Event publishing applications can publish data to geo-replicated namespaces via stable namespace FQDN of the geo replicated namespace. The event publishing approach is the same as the non-Geo DR case and no changes to client applications are required.
116
-
Event publishing may not be available during the following circumstances:
119
+
Event publishing might not be available during the following circumstances:
117
120
- During Failover grace period, the existing primary region rejects any new events that are published to Event Hubs.
118
121
- When replication lag between primary and secondary regions reaches the max replication lag duration, the publisher ingress workload may get throttled.
119
122
Publisher applications can't directly access any namespaces in the secondary regions.
@@ -134,4 +137,7 @@ Clients that use the Event Hubs SDK need to upgrade to the April 2024 version of
134
137
135
138
## Pricing
136
139
Event Hubs dedicated clusters are priced independently of geo-replication. Use of geo-replication with Event Hubs dedicated requires you to have at least two dedicated clusters in separate regions. The dedicated clusters used as secondary instances for geo-replication can be used for other workloads.
137
-
There is a charge for geo-replication based on the published bandwidth * the number of secondary regions. The geo-replication charge is waived in early public preview.
140
+
There's a charge for geo-replication based on the published bandwidth * the number of secondary regions. The geo-replication charge is waived in early public preview.
141
+
142
+
## Related content
143
+
To learn how to use the Geo-replication feature, see [Use Geo-replication](use-geo-replication.md).
Copy file name to clipboardExpand all lines: articles/event-hubs/use-geo-replication.md
+28-22Lines changed: 28 additions & 22 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,11 +2,14 @@
2
2
title: 'How to use Azure Event Hubs geo-replication'
3
3
description: 'This article describes how to use the Azure Event Hubs geo-replication feature'
4
4
ms.topic: article
5
+
author: axisc
6
+
ms.author: aschhabria
5
7
ms.date: 06/10/2024
6
8
---
7
9
# How to use Geo-replication (Public Preview)
8
10
9
11
This tutorial shows you how to use the Geo-replication with your Event Hubs Dedicated namespace. To learn more about this feature, read the Geo-replication article. In this article you learn how to:
12
+
10
13
- Enable Geo-replication on a new namespace.
11
14
- Enable Geo-replication on an existing namespace.
12
15
- Perform a planned promotion or failover.
@@ -20,51 +23,54 @@ To use the Geo-replication feature, you need to have at least one Dedicated Even
20
23
You can enable Geo-replication during namespace creation and after namespace creation.
21
24
To enable Geo-replication on a namespace during namespace creation:
22
25
23
-
1. Click on ‘Namespace’ to create a new Event Hubs namespace in an Event Hubs cluster in a region with Geo-replication enabled. Provide a name for the namespace and check the Enable Geo-replication box.
26
+
1. Navigate to the **Event Hubs Cluster** page for your Event Hubs cluster.
27
+
1. On the left menu, expand **Entities**, and select **Cluster Namespaces**.
28
+
1. To create an Event Hubs namespace in an Event Hubs cluster in a region with Geo-replication enabled, on the **Cluster Namespaces** page, on the toolbar, select **+ Namespace**. Provide a name for the namespace, and select **Enable Geo-replication**.
24
29
25
-
:::image type="content" source="./media/use-geo-replication/namespace-create.png" alt-text="Screenshot of dedicated namespace create UI with geo-replication UI.":::
26
-
27
-
2. Click on Add secondary region and select a secondary region and a corresponding Event Hubs Dedicated cluster running in that region.
30
+
:::image type="content" source="./media/use-geo-replication/namespace-create.png" alt-text="Screenshot of dedicated namespace create UI with geo-replication UI.":::
31
+
2. Select **Add secondary region**, and select a secondary region and a corresponding Event Hubs dedicated cluster running in that region.
28
32
29
-
:::image type="content" source="./media/use-geo-replication/region-selection.png" alt-text="Screenshot of secondary region and cluster selection in namespace create UI.":::
33
+
:::image type="content" source="./media/use-geo-replication/region-selection.png" alt-text="Screenshot of secondary region and cluster selection in namespace create UI.":::
30
34
31
-
3.Select asynchronous or synchronous replication mode as the replication consistency mode. If selecting asynchronous consistency, enter the allowable amount of time the secondary region can lag behind the primary region in minutes.
35
+
3.Select asynchronous or synchronous **replication mode** as the replication consistency mode. If you select asynchronous consistency, enter the allowable amount of time the secondary region can lag behind the primary region in minutes.
32
36
33
-
:::image type="content" source="./media/use-geo-replication/create-replication-consistency.png" alt-text="Screenshot of replication consistency UI in dedicated namespace create UI.":::
34
-
35
-
4. Then click on ‘Create’ to create the Geo Replicated Event Hubs namespace. The deployment takes a couple of minutes to complete.
36
-
5. Once the namespace is created, you can navigate to it and click on "Geo-replication" tab to see your Geo-replication configuration.
37
+
:::image type="content" source="./media/use-geo-replication/create-replication-consistency.png" alt-text="Screenshot of replication consistency UI in dedicated namespace create UI.":::
38
+
4. Then, select **Create** to create the Geo-replicated Event Hubs namespace. The deployment takes a couple of minutes to complete.
39
+
5. Once the namespace is created, you can navigate to it and select **Geo-replication** on the left menu to see your Geo-replication configuration.
37
40
38
-
:::image type="content" source="./media/use-geo-replication/geo-replication.png" alt-text="Screenshot of geo-replication UI that shows configuration and allows various actions.":::
41
+
:::image type="content" source="./media/use-geo-replication/geo-replication.png" alt-text="Screenshot of geo-replication UI that shows configuration and allows various actions.":::
39
42
40
43
## Enable Geo-replication on an existing namespace
41
-
1. Go into the namespace in the portal and click on "Geo-replication".
42
-
2. Click on Add secondary region and select a secondary region and the corresponding Event Hubs Dedicated clusters running in that region.
43
-
3. Select asynchronous or synchronous replication mode as the replication consistency mode. If selecting asynchronous consistency, enter the allowable amount of time the secondary region can lag behind the primary region in minutes.
44
44
45
-
:::image type="content" source="./media/use-geo-replication/geo-replication-consistency.png" alt-text="Screenshot of replication consistency UI in geo-replication UI.":::
45
+
1. Navigate to your Event Hubs namespace in the Azure portal, and select **Geo-replication** on the left menu.
46
+
2. Select **Add secondary region**, and select a secondary region and the corresponding Event Hubs Dedicated clusters running in that region.
47
+
3. Select asynchronous or synchronous replication mode as the replication consistency mode. If selecting asynchronous consistency, enter the allowable amount of time the secondary region can lag behind the primary region in minutes.
48
+
49
+
:::image type="content" source="./media/use-geo-replication/geo-replication-consistency.png" alt-text="Screenshot of replication consistency UI in geo-replication UI.":::
46
50
47
-
After a secondary region is added, all of the data held in the primary namespace is replicated to the secondary. Complete replication can take a while depending on various factors with the main one being how much data is in your primary namespace. Users can observe replication progress by monitoring the lag to the secondary region.
51
+
After a secondary region is added, all of the data held in the primary namespace is replicated to the secondary. Complete replication can take a while depending on various factors with the main one being how much data is in your primary namespace. Users can observe replication progress by monitoring the lag to the secondary region.
48
52
49
53
## Promote secondary
50
54
You can promote your configured secondary region to being the primary region. When you promote a secondary region to primary, the current primary region becomes the secondary region. A promotion can be planned or forced. Planned promotions ensure both regions are caught up before accepting new traffic. Forced promotions take effect as quickly as possible and doesn't wait for things to be caught up.
51
55
52
56
To initiate a promotion of your secondary region to primary, select failover icon.
53
57
54
-
:::image type="content" source="./media/use-geo-replication/promotion-a.png" alt-text="Screenshot of the promotion UI selection in the geo-replication UI.":::
58
+
:::image type="content" source="./media/use-geo-replication/promotion-a.png" alt-text="Screenshot of the promotion UI selection in the geo-replication UI.":::
55
59
56
-
When in the promotion flow, you can select planned or forced. You can also choose to select forced after starting a planned promotion. Enter the word "promote" in the prompt to be able to start the promotion.
60
+
When in the promotion flow, you can select planned or forced. You can also choose to select forced after starting a planned promotion. Enter the word **promote** in the prompt to be able to start the promotion.
57
61
58
-
:::image type="content" source="./media/use-geo-replication/promotion.png" alt-text="Screenshot of the promotion UI in where you can select planned or forced.":::
62
+
:::image type="content" source="./media/use-geo-replication/promotion.png" alt-text="Screenshot of the promotion UI in where you can select planned or forced.":::
59
63
60
-
If doing a planned promotion, then once the promotion process is initiated, the new primary rejects any new events until failover is completed. The promotion process repoints the fully qualified domain name(FQDN) for your namespace to the selected region, complete data replication between the two regions and configure the new primary region to be active. Promotion does not require any changes to clients, and that they continue to work after the promotion event.
64
+
If doing a planned promotion, then once the promotion process is initiated, the new primary rejects any new events until failover is completed. The promotion process repoints the fully qualified domain name(FQDN) for your namespace to the selected region, complete data replication between the two regions and configure the new primary region to be active. Promotion doesn't require any changes to clients, and that they continue to work after the promotion event.
61
65
62
66
In the case where your primary region goes down completely, you can still perform a forced promotion.
63
67
64
68
## Remove a secondary
65
-
To remove a Geo-replication pairing with a secondary, go into "Geo-replication", select the secondary region, and then select remove. At the prompt, enter the word "delete" and then you can delete the secondary.
69
+
To remove a Geo-replication pairing with a secondary, select **Geo-replication** on the left menu, select the secondary region, and then select **Remove**. At the prompt, enter the word **delete**, and then you can delete the secondary.
66
70
67
-
:::image type="content" source="./media/use-geo-replication/remove-secondary.png" alt-text="Screenshot of the remove secondary function in the geo-replcation UI.":::
71
+
:::image type="content" source="./media/use-geo-replication/remove-secondary.png" alt-text="Screenshot of the Remove secondary function in the geo-replcation UI.":::
68
72
69
73
When a secondary region is removed, all of the data that it held is also removed. If you wish to re-enable Geo-replication with that region and cluster, it has to replicate the primary region data all over again.
70
74
75
+
## Related content
76
+
For conceptual information about the Geo-replication feature, see [Azure Event Hubs geo-replication]geo-replication.md).
0 commit comments