You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/kafka/apache-kafka-mirroring.md
+14-14Lines changed: 14 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -41,7 +41,7 @@ If you need to mirror between Kafka clusters in different networks, there are th
41
41
***Domain names**: If you don't configure your Kafka clusters for IP address advertising, the clusters must be able to connect to each other by using fully qualified domain names (FQDNs). This requires a domain name system (DNS) server in each network that is configured to forward requests to the other networks. When you're creating an Azure virtual network, instead of using the automatic DNS provided with the network, you must specify a custom DNS server and the IP address for the server. After you create the virtual network, you must then create an Azure virtual machine that uses that IP address. Then you install and configure DNS software on it.
42
42
43
43
> [!IMPORTANT]
44
-
> Create and configure the custom DNS server before installing HDInsight into the virtual network. There is no additional configuration required for HDInsight to use the DNS server configured for the Virtual Network.
44
+
> Create and configure the custom DNS server before installing HDInsight into the virtual network. There is no additional configuration required for HDInsight to use the DNS server configured for the virtual network.
45
45
46
46
For more information on connecting two Azure virtual networks, see [Configure a connection](../../vpn-gateway/vpn-gateway-vnet-vnet-rm-ps.md).
47
47
@@ -105,7 +105,7 @@ Configure IP advertising to enable a client to connect by using broker IP addres
105
105
1. Select **Save**.
106
106
1. Select **Restart** > **Confirm Restart All**.
107
107
108
-
### Record broker IP addresses and ZooKeeper addresses for primary cluster
108
+
### Record broker IP addresses and ZooKeeper addresses for the primary cluster
109
109
110
110
1. Select **Hosts** on the Ambari dashboard.
111
111
1. Make a note of the IP addresses for the brokers and ZooKeepers. The broker nodes have **wn** as the first two letters of the host name, and the ZooKeeper nodes have **zk** as the first two letters of the host name.
@@ -116,13 +116,13 @@ Configure IP advertising to enable a client to connect by using broker IP addres
116
116
117
117
## Create topics
118
118
119
-
1. Connect to the **primary** cluster by using SSH:
Replace `sshuser` with the SSH user name used when creating the cluster. Replace `PRIMARYCLUSTER` with the base name used when creating the cluster.
125
+
Replace `sshuser` with the SSH user name that you used when creating the cluster. Replace `PRIMARYCLUSTER` with the base name that you used when creating the cluster.
126
126
127
127
For more information, see [Use SSH with HDInsight](../hdinsight-hadoop-linux-use-ssh-unix.md).
128
128
@@ -164,17 +164,17 @@ Configure IP advertising to enable a client to connect by using broker IP addres
164
164
165
165
## Configure mirroring
166
166
167
-
1. Connect to the **secondary** cluster by using a different SSH session:
167
+
1. Connect to the secondary cluster by using a different SSH session:
Replace `sshuser` with the SSH user name used when creating the cluster. Replace `SECONDARYCLUSTER` with the name used when creating the cluster.
173
+
Replace `sshuser` with the SSH user name that you used when creating the cluster. Replace `SECONDARYCLUSTER` with the name that you used when creating the cluster.
174
174
175
175
For more information, see [Use SSH with HDInsight](../hdinsight-hadoop-linux-use-ssh-unix.md).
176
176
177
-
1. Use a `consumer.properties` file to configure communication with the **primary** cluster. To create the file, use the following command:
177
+
1. Use a `consumer.properties` file to configure communication with the primary cluster. To create the file, use the following command:
178
178
179
179
```bash
180
180
nano consumer.properties
@@ -187,13 +187,13 @@ Configure IP advertising to enable a client to connect by using broker IP addres
187
187
group.id=mirrorgroup
188
188
```
189
189
190
-
Replace `PRIMARY_BROKERHOSTS` with the broker host IP addresses from the **primary** cluster.
190
+
Replace `PRIMARY_BROKERHOSTS` with the broker host IP addresses from the primary cluster.
191
191
192
192
This file describes the consumer information to use when reading from the primary Kafka cluster. For more information, see [Consumer Configs](https://kafka.apache.org/documentation#consumerconfigs) at `kafka.apache.org`.
193
193
194
194
To save the file, press Ctrl+X, press Y, and then press Enter.
195
195
196
-
1. Before configuring the producer that communicates with the secondary cluster, set up a variable for the broker IP addresses of the **secondary** cluster. Use the following commands to create this variable:
196
+
1. Before configuring the producer that communicates with the secondary cluster, set up a variable for the broker IP addresses of the secondary cluster. Use the following commands to create this variable:
@@ -203,7 +203,7 @@ Configure IP advertising to enable a client to connect by using broker IP addres
203
203
204
204
`10.23.0.14:9092,10.23.0.4:9092,10.23.0.12:9092`
205
205
206
-
1. Use a `producer.properties` file to communicate the **secondary** cluster. To create the file, use the following command:
206
+
1. Use a `producer.properties` file to communicate the secondary cluster. To create the file, use the following command:
207
207
208
208
```bash
209
209
nano producer.properties
@@ -256,7 +256,7 @@ Configure IP advertising to enable a client to connect by using broker IP addres
256
256
> [!NOTE]
257
257
> This article contains references to the term *whitelist*, a term that Microsoft no longer uses. When the term is removed from the software, we'll remove it from this article.
258
258
259
-
1. From the SSH connection to the **secondary** cluster, use the following command to start the MirrorMaker process:
259
+
1. From the SSH connection to the secondary cluster, use the following command to start the MirrorMaker process:
When you arrive at a blank line with a cursor, type in a few text messages. The messages are sent to the topic on the **primary** cluster. When done, press Ctrl+C to end the producer process.
282
+
When you arrive at a blank line with a cursor, type in a few text messages. The messages are sent to the topic on the primary cluster. When done, press Ctrl+C to end the producer process.
283
283
284
-
1. From the SSH connection to the **secondary** cluster, press Ctrl+C to end the MirrorMaker process. It might take several seconds to end the process. To verify that the messages were replicated to the secondary, use the following command:
284
+
1. From the SSH connection to the secondary cluster, press Ctrl+C to end the MirrorMaker process. It might take several seconds to end the process. To verify that the messages were replicated to the secondary, use the following command:
0 commit comments