You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/event-hubs/event-hubs-kafka-mirror-maker-tutorial.md
+28-12Lines changed: 28 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,16 +16,26 @@ ms.author: bahariri
16
16
17
17
# Use Kafka MirrorMaker with Event Hubs for Apache Kafka
18
18
19
+
This tutorial shows how to mirror a Kafka broker in a Kafka enabled event hub using Kafka MirrorMaker.
20
+
21
+

22
+
19
23
> [!NOTE]
20
24
> This sample is available on [GitHub](https://github.com/Azure/azure-event-hubs)
21
25
22
-
One major consideration for modern cloud scale apps is the ability to update, improve, and change infrastructure without interrupting service. This tutorial shows how a Kafka-enabled event hub and Kafka MirrorMaker can integrate an existing Kafka pipeline into Azure by "mirroring" the Kafka input stream in the Event Hubs service.
23
26
24
-
An Azure Event Hubs Kafka endpoint enables you to connect to Azure Event Hubs using the Kafka protocol (that is, Kafka clients). By making minimal changes to a Kafka application, you can connect to Azure Event Hubs and enjoy the benefits of the Azure ecosystem. Kafka enabled Event Hubs currently supports Kafka versions 1.0 and later.
27
+
In this tutorial, you learn how to:
28
+
> [!div class="checklist"]
29
+
> * Create an Event Hubs namespace
30
+
> * Clone the example project
31
+
> * Set up a Kafka cluster
32
+
> * Configure Kafka MirrorMaker
33
+
> * Run Kafka MirrorMaker
25
34
26
-
This example shows how to mirror a Kafka broker in a Kafka enabled event hub using Kafka MirrorMaker.
35
+
## Introduction
36
+
One major consideration for modern cloud scale apps is the ability to update, improve, and change infrastructure without interrupting service. This tutorial shows how a Kafka-enabled event hub and Kafka MirrorMaker can integrate an existing Kafka pipeline into Azure by "mirroring" the Kafka input stream in the Event Hubs service.
27
37
28
-

38
+
An Azure Event Hubs Kafka endpoint enables you to connect to Azure Event Hubs using the Kafka protocol (that is, Kafka clients). By making minimal changes to a Kafka application, you can connect to Azure Event Hubs and enjoy the benefits of the Azure ecosystem. Kafka enabled Event Hubs currently supports Kafka versions 1.0 and later.
29
39
30
40
## Prerequisites
31
41
@@ -57,14 +67,12 @@ cd azure-event-hubs/samples/kafka/mirror-maker
57
67
58
68
Use the [Kafka quickstart guide](https://kafka.apache.org/quickstart) to set up a cluster with the desired settings (or use an existing Kafka cluster).
59
69
60
-
## Kafka MirrorMaker
70
+
## Configure Kafka MirrorMaker
61
71
62
72
Kafka MirrorMaker enables the "mirroring" of a stream. Given source and destination Kafka clusters, MirrorMaker ensures any messages sent to the source cluster are received by both the source and destination clusters. This example shows how to mirror a source Kafka cluster with a destination Kafka-enabled event hub. This scenario can be used to send data from an existing Kafka pipeline to Event Hubs without interrupting the flow of data.
63
73
64
74
For more detailed information on Kafka MirrorMaker, see the [Kafka Mirroring/MirrorMaker guide](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27846330).
65
75
66
-
### Configuration
67
-
68
76
To configure Kafka MirrorMaker, give it a Kafka cluster as its consumer/source and a Kafka-enabled event hub as its producer/destination.
Run the Kafka MirrorMaker script from the root Kafka directory using the newly updated configuration files. Make sure to either copy the config files to the root Kafka directory, or update their paths in the following command.
102
110
@@ -110,7 +118,15 @@ With MirrorMaker running, any events sent to the source Kafka cluster are receiv
110
118
111
119
## Next steps
112
120
113
-
*[Learn about Event Hubs](event-hubs-what-is-event-hubs.md)
114
-
*[Learn about Event Hubs for Kafka](event-hubs-for-kafka-ecosystem-overview.md)
115
-
* Learn more about [MirrorMaker](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27846330) to stream events from Kafka on-premises to Kafka enabled event hubs on cloud.
116
-
* Learn how to stream into Kafka enabled Event Hubs using [native Kafka applications](event-hubs-quickstart-kafka-enabled-event-hubs.md), [Apache Flink](event-hubs-kafka-flink-tutorial.md), or [Akka Streams](event-hubs-kafka-akka-streams-tutorial.md).
121
+
In this tutorial, you learn how to:
122
+
> [!div class="checklist"]
123
+
> * Create an Event Hubs namespace
124
+
> * Clone the example project
125
+
> * Set up a Kafka cluster
126
+
> * Configure Kafka MirrorMaker
127
+
> * Run Kafka MirrorMaker
128
+
129
+
Advance to the next article to learn more about Event Hubs for Apache Kafka:
130
+
131
+
> [!div class="nextstepaction"]
132
+
> [Use Apache Flink with Azure Event Hubs for Kafka](event-hubs-kafka-flink-tutorial.md)
0 commit comments