Skip to content

Commit 2177ebe

Browse files
committed
freshness51
1 parent a6b3852 commit 2177ebe

5 files changed

+35
-33
lines changed

articles/hdinsight/hadoop/apache-hadoop-use-hive-beeline.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,8 +35,7 @@ When connecting from a client to HDInsight over an Azure Virtual Network, you mu
3535
beeline -u 'jdbc:hive2://<headnode-FQDN>:10001/;transportMode=http'
3636
```
3737

38-
Replace `<headnode-FQDN>` with the fully qualified domain name of a cluster headnode. To find the fully qualified domain name of a headnode, use the information in the [Manage HDInsight using the Apache Ambari REST API](../hdinsight-hadoop-manage-ambari-rest-api.md#example-get-the-fqdn-of-cluster-nodes) document.
39-
38+
Replace `<headnode-FQDN>` with the fully qualified domain name of a cluster headnode. To find the fully qualified domain name of a headnode, use the information in the [Manage HDInsight using the Apache Ambari REST API](../hdinsight-hadoop-manage-ambari-rest-api.md#get-the-fqdn-of-cluster-nodes) document.
4039
---
4140

4241
### To HDInsight Enterprise Security Package (ESP) cluster using Kerberos

articles/hdinsight/hdinsight-hadoop-linux-use-ssh-unix.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -173,7 +173,7 @@ The worker nodes and Zookeeper nodes aren't directly accessible from the interne
173173
ssh sshuser@wn0-myhdi
174174
```
175175

176-
To retrieve a list of the node names, see the [Manage HDInsight by using the Apache Ambari REST API](hdinsight-hadoop-manage-ambari-rest-api.md#example-get-the-fqdn-of-cluster-nodes) document.
176+
To retrieve a list of the node names, see the [Manage HDInsight by using the Apache Ambari REST API](hdinsight-hadoop-manage-ambari-rest-api.md#get-the-fqdn-of-cluster-nodes) document.
177177

178178
If the SSH account is secured using a __password__, enter the password when connecting.
179179

articles/hdinsight/hdinsight-hadoop-manage-ambari-rest-api.md

Lines changed: 31 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
---
22
title: Monitor and manage Hadoop with Ambari REST API - Azure HDInsight
3-
description: Learn how to use Ambari to monitor and manage Hadoop clusters in Azure HDInsight. In this document, you will learn how to use the Ambari REST API included with HDInsight clusters.
3+
description: Learn how to use Ambari to monitor and manage Hadoop clusters in Azure HDInsight. In this document, you'll learn how to use the Ambari REST API included with HDInsight clusters.
44
author: hrasheed-msft
55
ms.author: hrasheed
66
ms.reviewer: jasonh
77
ms.service: hdinsight
8-
ms.custom: hdinsightactive
98
ms.topic: conceptual
10-
ms.date: 06/07/2019
9+
ms.custom: hdinsightactive
10+
ms.date: 04/14/2020
1111
---
1212

1313
# Manage HDInsight clusters by using the Apache Ambari REST API
@@ -16,21 +16,21 @@ ms.date: 06/07/2019
1616

1717
Learn how to use the Apache Ambari REST API to manage and monitor Apache Hadoop clusters in Azure HDInsight.
1818

19-
## <a id="whatis"></a>What is Apache Ambari
19+
## What is Apache Ambari
2020

2121
[Apache Ambari](https://ambari.apache.org) simplifies the management and monitoring of Hadoop clusters by providing an easy to use web UI backed by its [REST APIs](https://github.com/apache/ambari/blob/trunk/ambari-server/docs/api/v1/index.md). Ambari is provided by default with Linux-based HDInsight clusters.
2222

2323
## Prerequisites
2424

25-
* **A Hadoop cluster on HDInsight**. See [Get Started with HDInsight on Linux](hadoop/apache-hadoop-linux-tutorial-get-started.md).
25+
* A Hadoop cluster on HDInsight. See [Get Started with HDInsight on Linux](hadoop/apache-hadoop-linux-tutorial-get-started.md).
2626

27-
* **Bash on Ubuntu on Windows 10**. The examples in this article use the Bash shell on Windows 10. See [Windows Subsystem for Linux Installation Guide for Windows 10](https://docs.microsoft.com/windows/wsl/install-win10) for installation steps. Other [Unix shells](https://www.gnu.org/software/bash/) will work as well. The examples, with some slight modifications, can work on a Windows Command prompt. Alternatively, you can use Windows PowerShell.
27+
* Bash on Ubuntu on Windows 10. The examples in this article use the Bash shell on Windows 10. See [Windows Subsystem for Linux Installation Guide for Windows 10](https://docs.microsoft.com/windows/wsl/install-win10) for installation steps. Other [Unix shells](https://www.gnu.org/software/bash/) will work as well. The examples, with some slight modifications, can work on a Windows Command prompt. Or you can use Windows PowerShell.
2828

29-
* **jq**, a command-line JSON processor. See [https://stedolan.github.io/jq/](https://stedolan.github.io/jq/).
29+
* jq, a command-line JSON processor. See [https://stedolan.github.io/jq/](https://stedolan.github.io/jq/).
3030

31-
* **Windows PowerShell**. Alternatively, you can use [Bash](https://www.gnu.org/software/bash/).
31+
* Windows PowerShell. Or you can use [Bash](https://www.gnu.org/software/bash/).
3232

33-
## Base URI for Ambari Rest API
33+
## Base Uniform Resource Identifier for Ambari Rest API
3434

3535
The base Uniform Resource Identifier (URI) for the Ambari REST API on HDInsight is `https://CLUSTERNAME.azurehdinsight.net/api/v1/clusters/CLUSTERNAME`, where `CLUSTERNAME` is the name of your cluster. Cluster names in URIs are **case-sensitive**. While the cluster name in the fully qualified domain name (FQDN) part of the URI (`CLUSTERNAME.azurehdinsight.net`) is case-insensitive, other occurrences in the URI are case-sensitive.
3636

@@ -43,6 +43,7 @@ For Enterprise Security Package clusters, instead of `admin`, use a fully qualif
4343
## Examples
4444

4545
### Setup (Preserve credentials)
46+
4647
Preserve your credentials to avoid reentering them for each example. The cluster name will be preserved in a separate step.
4748

4849
**A. Bash**
@@ -59,9 +60,10 @@ $creds = Get-Credential -UserName "admin" -Message "Enter the HDInsight login"
5960
```
6061

6162
### Identify correctly cased cluster name
62-
The actual casing of the cluster name may be different than you expect, depending on how the cluster was created. The steps here will show the actual casing, and then store it in a variable for all subsequent examples.
6363

64-
Edit the scripts below to replace `CLUSTERNAME` with your cluster name. Then enter the command. (The cluster name for the FQDN is not case-sensitive.)
64+
The actual casing of the cluster name may be different than you expect. The steps here will show the actual casing, and then store it in a variable for all later examples.
65+
66+
Edit the scripts below to replace `CLUSTERNAME` with your cluster name. Then enter the command. (The cluster name for the FQDN isn't case-sensitive.)
6567

6668
```bash
6769
export clusterName=$(curl -u admin:$password -sS -G "https://CLUSTERNAME.azurehdinsight.net/api/v1/clusters" | jq -r '.items[].Clusters.cluster_name')
@@ -94,9 +96,9 @@ $respObj = ConvertFrom-Json $resp.Content
9496
$respObj.Clusters.health_report
9597
```
9698

97-
### <a name="example-get-the-fqdn-of-cluster-nodes"></a> Get the FQDN of cluster nodes
99+
### Get the FQDN of cluster nodes
98100

99-
When working with HDInsight, you may need to know the fully qualified domain name (FQDN) of a cluster node. You can easily retrieve the FQDN for the various nodes in the cluster using the following examples:
101+
You may need to know the fully qualified domain name (FQDN) of a cluster node. You can easily retrieve the FQDN for the various nodes in the cluster using the following examples:
100102

101103
**All nodes**
102104

@@ -154,13 +156,13 @@ $respObj = ConvertFrom-Json $resp.Content
154156
$respObj.host_components.HostRoles.host_name
155157
```
156158

157-
### <a name="example-get-the-internal-ip-address-of-cluster-nodes"></a> Get the internal IP address of cluster nodes
159+
### Get the internal IP address of cluster nodes
158160

159-
The IP addresses returned by the examples in this section are not directly accessible over the internet. They are only accessible within the Azure Virtual Network that contains the HDInsight cluster.
161+
The IP addresses returned by the examples in this section aren't directly accessible over the internet. They're only accessible within the Azure Virtual Network that contains the HDInsight cluster.
160162

161163
For more information on working with HDInsight and virtual networks, see [Plan a virtual network for HDInsight](hdinsight-plan-virtual-network-deployment.md).
162164

163-
To find the IP address, you must know the internal fully qualified domain name (FQDN) of the cluster nodes. Once you have the FQDN, you can then get the IP address of the host. The following examples first query Ambari for the FQDN of all the host nodes, then query Ambari for the IP address of each host.
165+
To find the IP address, you must know the internal fully qualified domain name (FQDN) of the cluster nodes. Once you have the FQDN, you can then get the IP address of the host. The following examples first query Ambari for the FQDN of all the host nodes. Then queries Ambari for the IP address of each host.
164166

165167
```bash
166168
for HOSTNAME in $(curl -u admin:$password -sS -G "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/hosts" | jq -r '.items[].Hosts.host_name')
@@ -178,15 +180,15 @@ foreach($item in $respObj.items) {
178180
$hostName = [string]$item.Hosts.host_name
179181
$hostInfoResp = Invoke-WebRequest -Uri "$uri/$hostName" `
180182
-Credential $creds -UseBasicParsing
181-
$hostInfoObj = ConvertFrom-Json $hostInfoResp
183+
$hostInfoObj = ConvertFrom-Json $hostInfoResp
182184
$hostIp = $hostInfoObj.Hosts.ip
183185
"$hostName <--> $hostIp"
184186
}
185187
```
186188

187189
### Get the default storage
188190

189-
When you create an HDInsight cluster, you must use an Azure Storage Account or Data Lake Storage as the default storage for the cluster. You can use Ambari to retrieve this information after the cluster has been created. For example, if you want to read/write data to the container outside HDInsight.
191+
HDInsight clusters must use an Azure Storage Account or Data Lake Storage as the default storage. You can use Ambari to retrieve this information after the cluster has been created. For example, if you want to read/write data to the container outside HDInsight.
190192

191193
The following examples retrieve the default storage configuration from the cluster:
192194

@@ -248,7 +250,7 @@ The return value is similar to one of the following examples:
248250
> [!NOTE]
249251
> The [Get-AzHDInsightCluster](https://docs.microsoft.com/powershell/module/az.hdinsight/get-azhdinsightcluster) cmdlet provided by [Azure PowerShell](/powershell/azure/overview) also returns the storage information for the cluster.
250252

251-
### <a name="get-all-configurations"></a> Get all configurations
253+
### Get all configurations
252254

253255
Get the configurations that are available for your cluster.
254256

@@ -262,7 +264,7 @@ $respObj = Invoke-WebRequest -Uri "https://$clusterName.azurehdinsight.net/api/v
262264
$respObj.Content
263265
```
264266
265-
This example returns a JSON document containing the current configuration (identified by the *tag* value) for the components installed on the cluster. The following example is an excerpt from the data returned from a Spark cluster type.
267+
This example returns a JSON document containing the current configuration for installed components. See the *tag* value. The following example is an excerpt from the data returned from a Spark cluster type.
266268
267269
```json
268270
"jupyter-site" : {
@@ -281,7 +283,7 @@ This example returns a JSON document containing the current configuration (ident
281283
282284
### Get configuration for specific component
283285
284-
Get the configuration for the component that you are interested in. In the following example, replace `INITIAL` with the tag value returned from the previous request.
286+
Get the configuration for the component that you're interested in. In the following example, replace `INITIAL` with the tag value returned from the previous request.
285287
286288
```bash
287289
curl -u admin:$password -sS -G "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/configurations?type=livy2-conf&tag=INITIAL"
@@ -300,10 +302,11 @@ This example returns a JSON document containing the current configuration for th
300302
1. Create `newconfig.json`.
301303
Modify, and then enter the commands below:
302304
303-
* Replace `livy2-conf` with the desired component.
305+
* Replace `livy2-conf` with the new component.
304306
* Replace `INITIAL` with actual value retrieved for `tag` from [Get all configurations](#get-all-configurations).
305307
306-
**A. Bash**
308+
**A. Bash**
309+
307310
```bash
308311
curl -u admin:$password -sS -G "https://$clusterName.azurehdinsight.net/api/v1/clusters/$clusterName/configurations?type=livy2-conf&tag=INITIAL" \
309312
| jq --arg newtag $(echo version$(date +%s%N)) '.items[] | del(.href, .version, .Config) | .tag |= $newtag | {"Clusters": {"desired_config": .}}' > newconfig.json
@@ -321,11 +324,11 @@ This example returns a JSON document containing the current configuration for th
321324
$resp.Content | C:\HD\jq\jq-win64 --arg newtag "version$unixTimeStamp" '.items[] | del(.href, .version, .Config) | .tag |= $newtag | {"Clusters": {"desired_config": .}}' > newconfig.json
322325
```
323326
324-
Jq is used to turn the data retrieved from HDInsight into a new configuration template. Specifically, these examples perform the following actions:
327+
Jq is used to turn the data retrieved from HDInsight into a new configuration template. Specifically, these examples do the following actions:
325328
326329
* Creates a unique value containing the string "version" and the date, which is stored in `newtag`.
327330
328-
* Creates a root document for the new desired configuration.
331+
* Creates a root document for the new configuration.
329332
330333
* Gets the contents of the `.items[]` array and adds it under the **desired_config** element.
331334
@@ -358,7 +361,7 @@ This example returns a JSON document containing the current configuration for th
358361
359362
"livy.server.csrf_protection.enabled": "false",
360363
361-
Save the file once you are done making modifications.
364+
Save the file once you're done making modifications.
362365
363366
3. Submit `newconfig.json`.
364367
Use the following commands to submit the updated configuration to Ambari.
@@ -377,11 +380,11 @@ This example returns a JSON document containing the current configuration for th
377380
$resp.Content
378381
```
379382
380-
These commands submit the contents of the **newconfig.json** file to the cluster as the new desired configuration. The request returns a JSON document. The **versionTag** element in this document should match the version you submitted, and the **configs** object contains the configuration changes you requested.
383+
These commands submit the contents of the **newconfig.json** file to the cluster as the new configuration. The request returns a JSON document. The **versionTag** element in this document should match the version you submitted, and the **configs** object contains the configuration changes you requested.
381384
382385
### Restart a service component
383386
384-
At this point, if you look at the Ambari web UI, the Spark service indicates that it needs to be restarted before the new configuration can take effect. Use the following steps to restart the service.
387+
At this point, the Ambari web UI indicates the Spark service needs to be restarted before the new configuration can take effect. Use the following steps to restart the service.
385388
386389
1. Use the following to enable maintenance mode for the Spark2 service:
387390

articles/hdinsight/hdinsight-high-availability-linux.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ Nodes in an HDInsight cluster are implemented using Azure Virtual Machines. The
2828

2929
To ensure high availability of Hadoop services, HDInsight provides two head nodes. Both head nodes are active and running within the HDInsight cluster simultaneously. Some services, such as Apache HDFS or Apache Hadoop YARN, are only 'active' on one head node at any given time. Other services such as HiveServer2 or Hive MetaStore are active on both head nodes at the same time.
3030

31-
To obtain the hostnames for different node types in your cluster, please use the [Ambari REST API](hdinsight-hadoop-manage-ambari-rest-api.md#example-get-the-fqdn-of-cluster-nodes).
31+
To obtain the hostnames for different node types in your cluster, please use the [Ambari REST API](hdinsight-hadoop-manage-ambari-rest-api.md#get-the-fqdn-of-cluster-nodes).
3232

3333
> [!IMPORTANT]
3434
> Do not associate the numeric value with whether a node is primary or secondary. The numeric value is only present to provide a unique name for each node.

articles/hdinsight/kafka/apache-kafka-azure-container-services.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -158,7 +158,7 @@ At this point, Kafka and Azure Kubernetes Service are in communication through t
158158
* `var topic = 'mytopic'`: Replace `mytopic` with the name of the Kafka topic used by this application.
159159
* `var brokerHost = '176.16.0.13:9092`: Replace `176.16.0.13` with the internal IP address of one of the broker hosts for your cluster.
160160
161-
To find the internal IP address of the broker hosts (workernodes) in the cluster, see the [Apache Ambari REST API](../hdinsight-hadoop-manage-ambari-rest-api.md#example-get-the-internal-ip-address-of-cluster-nodes) document. Pick IP address of one of the entries where the domain name begins with `wn`.
161+
To find the internal IP address of the broker hosts (workernodes) in the cluster, see the [Apache Ambari REST API](../hdinsight-hadoop-manage-ambari-rest-api.md#get-the-internal-ip-address-of-cluster-nodes) document. Pick IP address of one of the entries where the domain name begins with `wn`.
162162
163163
4. From a command line in the `src` directory, install dependencies and use Docker to build an image for deployment:
164164

0 commit comments

Comments
 (0)