Skip to content

Commit 9d559fc

Browse files
authored
Merge pull request #90312 from dagiro/devops3
devops3
2 parents 9e94aa6 + 4f1e9ed commit 9d559fc

File tree

1 file changed

+24
-19
lines changed

1 file changed

+24
-19
lines changed

articles/hdinsight/kafka/apache-kafka-get-started.md

Lines changed: 24 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ ms.reviewer: jasonh
77
ms.service: hdinsight
88
ms.custom: mvc
99
ms.topic: quickstart
10-
ms.date: 06/12/2019
10+
ms.date: 10/01/2019
1111
#Customer intent: I need to create a Kafka cluster so that I can use it to process streaming data
1212
---
1313

@@ -107,24 +107,24 @@ To create an Apache Kafka on HDInsight cluster, use the following steps:
107107
3. When prompted, enter the password for the SSH user.
108108

109109
Once connected, you see information similar to the following text:
110-
110+
111111
```output
112112
Authorized uses only. All activity may be monitored and reported.
113113
Welcome to Ubuntu 16.04.4 LTS (GNU/Linux 4.13.0-1011-azure x86_64)
114-
114+
115115
* Documentation: https://help.ubuntu.com
116116
* Management: https://landscape.canonical.com
117117
* Support: https://ubuntu.com/advantage
118-
118+
119119
Get cloud support with Ubuntu Advantage Cloud Guest:
120120
https://www.ubuntu.com/business/services/cloud
121-
121+
122122
83 packages can be updated.
123123
37 updates are security updates.
124124
125125
126126
Welcome to Apache Kafka on HDInsight.
127-
127+
128128
Last login: Thu Mar 29 13:25:27 2018 from 108.252.109.241
129129
```
130130

@@ -140,29 +140,31 @@ In this section, you get the host information from the Apache Ambari REST API on
140140
sudo apt -y install jq
141141
```
142142

143-
2. Set up environment variables. Replace `PASSWORD` and `CLUSTERNAME` with the cluster login password and cluster name respectively, then enter the command:
143+
1. Set up password variable. Replace `PASSWORD` with the cluster login password, then enter the command:
144144

145145
```bash
146146
export password='PASSWORD'
147-
export clusterNameA='CLUSTERNAME'
148147
```
149148

150-
3. Extract correctly cased cluster name. The actual casing of the cluster name may be different than you expect, depending on how the cluster was created. This command will obtain the actual casing, store it in a variable, and then display the correctly cased name, and the name you provided earlier. Enter the following command:
149+
1. Extract the correctly cased cluster name. The actual casing of the cluster name may be different than you expect, depending on how the cluster was created. This command will obtain the actual casing, and then store it in a variable. Enter the following command:
151150

152151
```bash
153-
export clusterName=$(curl -u admin:$password -sS -G "https://$clusterNameA.azurehdinsight.net/api/v1/clusters" | jq -r '.items[].Clusters.cluster_name')
154-
echo $clusterName, $clusterNameA
152+
export clusterName=$(curl -u admin:$password -sS -G "http://headnodehost:8080/api/v1/clusters" | jq -r '.items[].Clusters.cluster_name')
155153
```
154+
> [!Note]
155+
> If you're running this command from outside the cluster, get the cluster name from the Azure portal. Enter clusterName in lowercase and run the command `export clusterName='clustername'` to store the variable.
156+
156157
157-
4. To set an environment variable with Zookeeper host information, use the command below. The command retrieves all Zookeeper hosts, then returns only the first two entries. This is because you want some redundancy in case one host is unreachable.
158+
1. To set an environment variable with Zookeeper host information, use the command below. The command retrieves all Zookeeper hosts, then returns only the first two entries. This is because you want some redundancy in case one host is unreachable.
158159
159160
```bash
160-
export KAFKAZKHOSTS=`curl -sS -u admin:$password -G http://headnodehost:8080/api/v1/clusters/$clusterName/services/ZOOKEEPER/components/ZOOKEEPER_SERVER | jq -r '["\(.host_components[].HostRoles.host_name):2181"] | join(",")' | cut -d',' -f1,2`
161+
export KAFKAZKHOSTS=$(curl -sS -u admin:$password -G https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/services/ZOOKEEPER/components/ZOOKEEPER_SERVER | jq -r '["\(.host_components[].HostRoles.host_name):2181"] | join(",")' | cut -d',' -f1,2);
161162
```
162163
163-
This command directly queries the Ambari service on the cluster head node. You can also access Ambari using the public address of `https://$CLUSTERNAME.azurehdinsight.net:80/`. Some network configurations can prevent access to the public address. For example, using Network Security Groups (NSG) to restrict access to HDInsight in a virtual network.
164+
> [!Note]
165+
> This command requires Ambari access. If your cluster is behind an NSG, run this command from a machine that can access Ambari.
164166
165-
5. To verify that the environment variable is set correctly, use the following command:
167+
1. To verify that the environment variable is set correctly, use the following command:
166168
167169
```bash
168170
echo $KAFKAZKHOSTS
@@ -172,15 +174,18 @@ In this section, you get the host information from the Apache Ambari REST API on
172174
173175
`zk0-kafka.eahjefxxp1netdbyklgqj5y1ud.ex.internal.cloudapp.net:2181,zk2-kafka.eahjefxxp1netdbyklgqj5y1ud.ex.internal.cloudapp.net:2181`
174176
175-
6. To set an environment variable with Apache Kafka broker host information, use the following command:
177+
1. To set an environment variable with Apache Kafka broker host information, use the following command:
176178
177179
```bash
178-
export KAFKABROKERS=`curl -sS -u admin:$password -G http://headnodehost:8080/api/v1/clusters/$clusterName/services/KAFKA/components/KAFKA_BROKER | jq -r '["\(.host_components[].HostRoles.host_name):9092"] | join(",")' | cut -d',' -f1,2`
180+
export KAFKABROKERS=$(curl -sS -u admin:$password -G https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/services/KAFKA/components/KAFKA_BROKER | jq -r '["\(.host_components[].HostRoles.host_name):9092"] | join(",")' | cut -d',' -f1,2);
179181
```
180182
181-
7. To verify that the environment variable is set correctly, use the following command:
183+
> [!Note]
184+
> This command requires Ambari access. If your cluster is behind an NSG, run this command from a machine that can access Ambari.
182185
183-
```bash
186+
1. To verify that the environment variable is set correctly, use the following command:
187+
188+
```bash
184189
echo $KAFKABROKERS
185190
```
186191

0 commit comments

Comments
 (0)