You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Replace `<headnode-FQDN>` with the fully qualified domain name of a cluster headnode. To find the fully qualified domain name of a headnode, use the information in the [Manage HDInsight using the Apache Ambari REST API](../hdinsight-hadoop-manage-ambari-rest-api.md#example-get-the-fqdn-of-cluster-nodes) document.
39
-
38
+
Replace `<headnode-FQDN>` with the fully qualified domain name of a cluster headnode. To find the fully qualified domain name of a headnode, use the information in the [Manage HDInsight using the Apache Ambari REST API](../hdinsight-hadoop-manage-ambari-rest-api.md#get-the-fqdn-of-cluster-nodes) document.
40
39
---
41
40
42
41
### To HDInsight Enterprise Security Package (ESP) cluster using Kerberos
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-hadoop-linux-use-ssh-unix.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -173,7 +173,7 @@ The worker nodes and Zookeeper nodes aren't directly accessible from the interne
173
173
ssh sshuser@wn0-myhdi
174
174
```
175
175
176
-
To retrieve a list of the node names, see the [Manage HDInsight by using the Apache Ambari REST API](hdinsight-hadoop-manage-ambari-rest-api.md#example-get-the-fqdn-of-cluster-nodes) document.
176
+
To retrieve a list of the node names, see the [Manage HDInsight by using the Apache Ambari REST API](hdinsight-hadoop-manage-ambari-rest-api.md#get-the-fqdn-of-cluster-nodes) document.
177
177
178
178
If the SSH account is secured using a __password__, enter the password when connecting.
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-hadoop-manage-ambari-rest-api.md
+31-28Lines changed: 31 additions & 28 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,13 +1,13 @@
1
1
---
2
2
title: Monitor and manage Hadoop with Ambari REST API - Azure HDInsight
3
-
description: Learn how to use Ambari to monitor and manage Hadoop clusters in Azure HDInsight. In this document, you will learn how to use the Ambari REST API included with HDInsight clusters.
3
+
description: Learn how to use Ambari to monitor and manage Hadoop clusters in Azure HDInsight. In this document, you'll learn how to use the Ambari REST API included with HDInsight clusters.
4
4
author: hrasheed-msft
5
5
ms.author: hrasheed
6
6
ms.reviewer: jasonh
7
7
ms.service: hdinsight
8
-
ms.custom: hdinsightactive
9
8
ms.topic: conceptual
10
-
ms.date: 06/07/2019
9
+
ms.custom: hdinsightactive
10
+
ms.date: 04/14/2020
11
11
---
12
12
13
13
# Manage HDInsight clusters by using the Apache Ambari REST API
@@ -16,21 +16,21 @@ ms.date: 06/07/2019
16
16
17
17
Learn how to use the Apache Ambari REST API to manage and monitor Apache Hadoop clusters in Azure HDInsight.
18
18
19
-
## <aid="whatis"></a>What is Apache Ambari
19
+
## What is Apache Ambari
20
20
21
21
[Apache Ambari](https://ambari.apache.org) simplifies the management and monitoring of Hadoop clusters by providing an easy to use web UI backed by its [REST APIs](https://github.com/apache/ambari/blob/trunk/ambari-server/docs/api/v1/index.md). Ambari is provided by default with Linux-based HDInsight clusters.
22
22
23
23
## Prerequisites
24
24
25
-
***A Hadoop cluster on HDInsight**. See [Get Started with HDInsight on Linux](hadoop/apache-hadoop-linux-tutorial-get-started.md).
25
+
* A Hadoop cluster on HDInsight. See [Get Started with HDInsight on Linux](hadoop/apache-hadoop-linux-tutorial-get-started.md).
26
26
27
-
***Bash on Ubuntu on Windows 10**. The examples in this article use the Bash shell on Windows 10. See [Windows Subsystem for Linux Installation Guide for Windows 10](https://docs.microsoft.com/windows/wsl/install-win10) for installation steps. Other [Unix shells](https://www.gnu.org/software/bash/) will work as well. The examples, with some slight modifications, can work on a Windows Command prompt. Alternatively, you can use Windows PowerShell.
27
+
* Bash on Ubuntu on Windows 10. The examples in this article use the Bash shell on Windows 10. See [Windows Subsystem for Linux Installation Guide for Windows 10](https://docs.microsoft.com/windows/wsl/install-win10) for installation steps. Other [Unix shells](https://www.gnu.org/software/bash/) will work as well. The examples, with some slight modifications, can work on a Windows Command prompt. Or you can use Windows PowerShell.
28
28
29
-
***jq**, a command-line JSON processor. See [https://stedolan.github.io/jq/](https://stedolan.github.io/jq/).
29
+
*jq, a command-line JSON processor. See [https://stedolan.github.io/jq/](https://stedolan.github.io/jq/).
30
30
31
-
***Windows PowerShell**. Alternatively, you can use [Bash](https://www.gnu.org/software/bash/).
31
+
* Windows PowerShell. Or you can use [Bash](https://www.gnu.org/software/bash/).
32
32
33
-
## Base URI for Ambari Rest API
33
+
## Base Uniform Resource Identifier for Ambari Rest API
34
34
35
35
The base Uniform Resource Identifier (URI) for the Ambari REST API on HDInsight is `https://CLUSTERNAME.azurehdinsight.net/api/v1/clusters/CLUSTERNAME`, where `CLUSTERNAME` is the name of your cluster. Cluster names in URIs are **case-sensitive**. While the cluster name in the fully qualified domain name (FQDN) part of the URI (`CLUSTERNAME.azurehdinsight.net`) is case-insensitive, other occurrences in the URI are case-sensitive.
36
36
@@ -43,6 +43,7 @@ For Enterprise Security Package clusters, instead of `admin`, use a fully qualif
43
43
## Examples
44
44
45
45
### Setup (Preserve credentials)
46
+
46
47
Preserve your credentials to avoid reentering them for each example. The cluster name will be preserved in a separate step.
The actual casing of the cluster name may be different than you expect, depending on how the cluster was created. The steps here will show the actual casing, and then store it in a variable for all subsequent examples.
63
63
64
-
Edit the scripts below to replace `CLUSTERNAME` with your cluster name. Then enter the command. (The cluster name for the FQDN is not case-sensitive.)
64
+
The actual casing of the cluster name may be different than you expect. The steps here will show the actual casing, and then store it in a variable for all later examples.
65
+
66
+
Edit the scripts below to replace `CLUSTERNAME` with your cluster name. Then enter the command. (The cluster name for the FQDN isn't case-sensitive.)
### <aname="example-get-the-fqdn-of-cluster-nodes"></a> Get the FQDN of cluster nodes
99
+
### Get the FQDN of cluster nodes
98
100
99
-
When working with HDInsight, you may need to know the fully qualified domain name (FQDN) of a cluster node. You can easily retrieve the FQDN for the various nodes in the cluster using the following examples:
101
+
You may need to know the fully qualified domain name (FQDN) of a cluster node. You can easily retrieve the FQDN for the various nodes in the cluster using the following examples:
### <aname="example-get-the-internal-ip-address-of-cluster-nodes"></a> Get the internal IP address of cluster nodes
159
+
### Get the internal IP address of cluster nodes
158
160
159
-
The IP addresses returned by the examples in this section are not directly accessible over the internet. They are only accessible within the Azure Virtual Network that contains the HDInsight cluster.
161
+
The IP addresses returned by the examples in this section aren't directly accessible over the internet. They're only accessible within the Azure Virtual Network that contains the HDInsight cluster.
160
162
161
163
For more information on working with HDInsight and virtual networks, see [Plan a virtual network for HDInsight](hdinsight-plan-virtual-network-deployment.md).
162
164
163
-
To find the IP address, you must know the internal fully qualified domain name (FQDN) of the cluster nodes. Once you have the FQDN, you can then get the IP address of the host. The following examples first query Ambari for the FQDN of all the host nodes, then query Ambari for the IP address of each host.
165
+
To find the IP address, you must know the internal fully qualified domain name (FQDN) of the cluster nodes. Once you have the FQDN, you can then get the IP address of the host. The following examples first query Ambari for the FQDN of all the host nodes. Then queries Ambari for the IP address of each host.
When you create an HDInsight cluster, you must use an Azure Storage Account or Data Lake Storage as the default storage for the cluster. You can use Ambari to retrieve this information after the cluster has been created. For example, if you want to read/write data to the container outside HDInsight.
191
+
HDInsight clusters must use an Azure Storage Account or Data Lake Storage as the default storage. You can use Ambari to retrieve this information after the cluster has been created. For example, if you want to read/write data to the container outside HDInsight.
190
192
191
193
The following examples retrieve the default storage configuration from the cluster:
192
194
@@ -248,7 +250,7 @@ The return value is similar to one of the following examples:
248
250
> [!NOTE]
249
251
> The [Get-AzHDInsightCluster](https://docs.microsoft.com/powershell/module/az.hdinsight/get-azhdinsightcluster) cmdlet provided by [Azure PowerShell](/powershell/azure/overview) also returns the storage information for the cluster.
250
252
251
-
### <a name="get-all-configurations"></a> Get all configurations
253
+
### Get all configurations
252
254
253
255
Get the configurations that are available for your cluster.
This example returns a JSON document containing the current configuration (identified by the *tag* value) for the components installed on the cluster. The following example is an excerpt from the data returned from a Spark cluster type.
267
+
This example returns a JSON document containing the current configuration for installed components. See the *tag* value. The following example is an excerpt from the data returned from a Spark cluster type.
266
268
267
269
```json
268
270
"jupyter-site": {
@@ -281,7 +283,7 @@ This example returns a JSON document containing the current configuration (ident
281
283
282
284
### Get configuration for specific component
283
285
284
-
Get the configuration for the component that you are interested in. In the following example, replace `INITIAL` with the tag value returned from the previous request.
286
+
Get the configuration for the component that you're interested in. In the following example, replace `INITIAL` with the tag value returned from the previous request.
Jq is used to turn the data retrieved from HDInsight into a new configuration template. Specifically, these examples perform the following actions:
327
+
Jq is used to turn the data retrieved from HDInsight into a new configuration template. Specifically, these examples do the following actions:
325
328
326
329
* Creates a unique value containing the string "version" and the date, which is stored in`newtag`.
327
330
328
-
* Creates a root document for the new desired configuration.
331
+
* Creates a root document for the new configuration.
329
332
330
333
* Gets the contents of the `.items[]` array and adds it under the **desired_config** element.
331
334
@@ -358,7 +361,7 @@ This example returns a JSON document containing the current configuration for th
358
361
359
362
"livy.server.csrf_protection.enabled": "false",
360
363
361
-
Save the file once you are done making modifications.
364
+
Save the file once you'redone making modifications.
362
365
363
366
3. Submit `newconfig.json`.
364
367
Use the following commands to submit the updated configuration to Ambari.
@@ -377,11 +380,11 @@ This example returns a JSON document containing the current configuration for th
377
380
$resp.Content
378
381
```
379
382
380
-
These commands submit the contents of the **newconfig.json** file to the cluster as the new desired configuration. The request returns a JSON document. The **versionTag** element in this document should match the version you submitted, and the **configs** object contains the configuration changes you requested.
383
+
These commands submit the contents of the **newconfig.json** file to the cluster as the new configuration. The request returns a JSON document. The **versionTag** element in this document should match the version you submitted, and the **configs** object contains the configuration changes you requested.
381
384
382
385
### Restart a service component
383
386
384
-
At this point, if you look at the Ambari web UI, the Spark service indicates that it needs to be restarted before the new configuration can take effect. Use the following steps to restart the service.
387
+
At this point, the Ambari web UI indicates the Spark service needs to be restarted before the new configuration can take effect. Use the following steps to restart the service.
385
388
386
389
1. Use the following to enable maintenance mode for the Spark2 service:
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-high-availability-linux.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,7 +28,7 @@ Nodes in an HDInsight cluster are implemented using Azure Virtual Machines. The
28
28
29
29
To ensure high availability of Hadoop services, HDInsight provides two head nodes. Both head nodes are active and running within the HDInsight cluster simultaneously. Some services, such as Apache HDFS or Apache Hadoop YARN, are only 'active' on one head node at any given time. Other services such as HiveServer2 or Hive MetaStore are active on both head nodes at the same time.
30
30
31
-
To obtain the hostnames for different node types in your cluster, please use the [Ambari REST API](hdinsight-hadoop-manage-ambari-rest-api.md#example-get-the-fqdn-of-cluster-nodes).
31
+
To obtain the hostnames for different node types in your cluster, please use the [Ambari REST API](hdinsight-hadoop-manage-ambari-rest-api.md#get-the-fqdn-of-cluster-nodes).
32
32
33
33
> [!IMPORTANT]
34
34
> Do not associate the numeric value with whether a node is primary or secondary. The numeric value is only present to provide a unique name for each node.
Copy file name to clipboardExpand all lines: articles/hdinsight/kafka/apache-kafka-azure-container-services.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -158,7 +158,7 @@ At this point, Kafka and Azure Kubernetes Service are in communication through t
158
158
*`var topic = 'mytopic'`: Replace `mytopic` with the name of the Kafka topic used by this application.
159
159
*`var brokerHost = '176.16.0.13:9092`: Replace `176.16.0.13` with the internal IP address of one of the broker hosts for your cluster.
160
160
161
-
To find the internal IP address of the broker hosts (workernodes) in the cluster, see the [Apache Ambari REST API](../hdinsight-hadoop-manage-ambari-rest-api.md#example-get-the-internal-ip-address-of-cluster-nodes) document. Pick IP address of one of the entries where the domain name begins with `wn`.
161
+
To find the internal IP address of the broker hosts (workernodes) in the cluster, see the [Apache Ambari REST API](../hdinsight-hadoop-manage-ambari-rest-api.md#get-the-internal-ip-address-of-cluster-nodes) document. Pick IP address of one of the entries where the domain name begins with `wn`.
162
162
163
163
4. From a command line in the `src` directory, install dependencies and use Docker to build an image for deployment:
0 commit comments