You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/domain-joined/apache-domain-joined-run-hive.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -103,7 +103,7 @@ In the last section, you configured two policies: `hiveuser1` has the select per
103
103
104
104
1. For the first use, an **ODBC driver** dialog opens. Select **Windows** from the left menu. Then select **Connect** to open the **Navigator** window.
105
105
106
-
1. Wait for the **Select Database and Table** dialog to open. This step can take a few seconds.
106
+
1. Wait for the `Select Database and Table` dialog to open. This step can take a few seconds.
107
107
108
108
1. Select **hivesampletable** > **Next**.
109
109
@@ -146,7 +146,7 @@ To test the second policy (read-hivesampletable-devicemake) that you created in
146
146
SELECT clientid, devicemake FROM "HIVE"."default"."hivesampletable"
147
147
```
148
148
149
-
When it's finished, you see two columns of imported data.
149
+
When finished, you see two columns of imported data.
Learn how to submit MapReduce jobs using HDInsight .NET SDK. HDInsight clusters come with a jar file with some MapReduce samples. The jar file is`/example/jars/hadoop-mapreduce-examples.jar`. One of the samples is **wordcount**. You develop a C# console application to submit a wordcount job. The job reads the `/example/data/gutenberg/davinci.txt` file, and outputs the results to `/example/data/davinciwordcount`. If you want to rerun the application, you must clean up the output folder.
14
+
Learn how to submit MapReduce jobs using HDInsight .NET SDK. HDInsight clusters come with a jar file with some MapReduce samples. The jar file is`/example/jars/hadoop-mapreduce-examples.jar`. One of the samples is **wordcount**. You develop a C# console application to submit a wordcount job. The job reads the `/example/data/gutenberg/davinci.txt` file, and outputs the results to `/example/data/davinciwordcount`. If you want to rerun the application, you must clean up the output folder.
15
15
16
16
> [!NOTE]
17
17
> The steps in this article must be performed from a Windows client. For information on using a Linux, OS X, or Unix client to work with Hive, use the tab selector shown on the top of the article.
@@ -34,7 +34,7 @@ The HDInsight .NET SDK provides .NET client libraries, which make it easier to w
1. Copy the code below into **Program.cs**. Then edit the code by setting the values for: `existingClusterName`, `existingClusterPassword`, `defaultStorageAccountName`, `defaultStorageAccountKey`, and `defaultStorageContainerName`.
37
+
1. Copy the code into **Program.cs**. Then edit the code by setting the values for: `existingClusterName`, `existingClusterPassword`, `defaultStorageAccountName`, `defaultStorageAccountKey`, and `defaultStorageContainerName`.
38
38
39
39
```csharp
40
40
using System.Collections.Generic;
@@ -155,13 +155,13 @@ The HDInsight .NET SDK provides .NET client libraries, which make it easier to w
155
155
156
156
1. Press **F5** to run the application.
157
157
158
-
To run the job again, you must change the job output folder name, in the sample it's `/example/data/davinciwordcount`.
158
+
To run the job again, you must change the job output folder name, in the sample its `/example/data/davinciwordcount`.
159
159
160
160
When the job completes successfully, the application prints the content of the output file `part-r-00000`.
161
161
162
162
## Next steps
163
163
164
-
In this article, you have learned several ways to create an HDInsight cluster. To learn more, see the following articles:
164
+
In this article, you learned several ways to create an HDInsight cluster. To learn more, see the following articles:
165
165
166
166
* For submitting a Hive job, see [Run Apache Hive queries using HDInsight .NET SDK](apache-hadoop-use-hive-dotnet-sdk.md).
167
167
* For creating HDInsight clusters, see [Create Linux-based Apache Hadoop clusters in HDInsight](../hdinsight-hadoop-provision-linux-clusters.md).
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-hadoop-manage-ambari-rest-api.md
+15-15Lines changed: 15 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
---
2
2
title: Monitor and manage Hadoop with Ambari REST API - Azure HDInsight
3
-
description: Learn how to use Ambari to monitor and manage Hadoop clusters in Azure HDInsight. In this document, you'll learn how to use the Ambari REST API included with HDInsight clusters.
3
+
description: Learn how to use Ambari to monitor and manage Hadoop clusters in Azure HDInsight. In this document, you learn how to use the Ambari REST API included with HDInsight clusters.
4
4
ms.service: hdinsight
5
5
ms.topic: how-to
6
6
ms.custom: hdinsightactive
@@ -21,7 +21,7 @@ Apache Ambari simplifies the management and monitoring of Hadoop clusters by pro
21
21
22
22
* A Hadoop cluster on HDInsight. See [Get Started with HDInsight on Linux](hadoop/apache-hadoop-linux-tutorial-get-started.md).
23
23
24
-
* Bash on Ubuntu on Windows 10. The examples in this article use the Bash shell on Windows 10. See [Windows Subsystem for Linux Installation Guide for Windows 10](/windows/wsl/install-win10) for installation steps. Other [Unix shells](https://www.gnu.org/software/bash/)will work as well. The examples, with some slight modifications, can work on a Windows Command prompt. Or you can use Windows PowerShell.
24
+
* Bash on Ubuntu on Windows 10. The examples in this article use the Bash shell on Windows 10. See [Windows Subsystem for Linux Installation Guide for Windows 10](/windows/wsl/install-win10) for installation steps. Other [Unix shells](https://www.gnu.org/software/bash/)works as well. The examples, with some slight modifications, can work on a Windows Command prompt. Or you can use Windows PowerShell.
25
25
26
26
* jq, a command-line JSON processor. See [https://stedolan.github.io/jq/](https://stedolan.github.io/jq/).
27
27
@@ -41,10 +41,10 @@ For Enterprise Security Package clusters, instead of `admin`, use a fully qualif
41
41
42
42
### Setup (Preserve credentials)
43
43
44
-
Preserve your credentials to avoid reentering them for each example. The cluster name will be preserved in a separate step.
44
+
Preserve your credentials to avoid reentering them for each example. The cluster name preserved in a separate step.
45
45
46
46
**A. Bash**
47
-
Edit the script below by replacing `PASSWORD` with your actual password. Then enter the command.
47
+
Edit the script by replacing `PASSWORD` with your actual password. Then enter the command.
The actual casing of the cluster name may be different than you expect. The steps here will show the actual casing, and then store it in a variable for all later examples.
61
+
The actual casing of the cluster name may be different than you expect. The following steps show the actual casing, and then store it in a variable for all later examples.
62
62
63
-
Edit the scripts below to replace `CLUSTERNAME` with your cluster name. Then enter the command. (The cluster name for the FQDN isn't case-sensitive.)
63
+
Edit the scripts to replace `CLUSTERNAME` with your cluster name. Then enter the command. (The cluster name for the FQDN isn't case-sensitive.)
@@ -185,7 +185,7 @@ foreach($item in $respObj.items) {
185
185
186
186
### Get the default storage
187
187
188
-
HDInsight clusters must use an Azure Storage Account or Data Lake Storage as the default storage. You can use Ambari to retrieve this information after the cluster has been created. For example, if you want to read/write data to the container outside HDInsight.
188
+
HDInsight clusters must use an Azure Storage Account or Data Lake Storage as the default storage. You can use Ambari to retrieve this information after the cluster created. For example, if you want to read/write data to the container outside HDInsight.
189
189
190
190
The following examples retrieve the default storage configuration from the cluster:
> These examples return the first configuration applied to the server (`service_config_version=1`) which contains this information. If you retrieve a value that has been modified after cluster creation, you may need to list the configuration versions and retrieve the latest one.
205
+
> These examples return the first configuration applied to the server (`service_config_version=1`) which contains this information. If you retrieve a value that modified after cluster creation, you may need to list the configuration versions and retrieve the latest one.
206
206
207
207
The return value is similar to one of the following examples:
208
208
@@ -297,7 +297,7 @@ This example returns a JSON document containing the current configuration for th
297
297
### Update configuration
298
298
299
299
1. Create `newconfig.json`.
300
-
Modify, and then enter the commands below:
300
+
Modify, and then enter the commands as follows:
301
301
302
302
* Replace `livy2-conf` with the new component.
303
303
* Replace `INITIAL` with actual value retrieved for`tag` from [Get all configurations](#get-all-configurations).
@@ -310,7 +310,7 @@ This example returns a JSON document containing the current configuration for th
310
310
```
311
311
312
312
**B. PowerShell**
313
-
The PowerShell script uses [jq](https://stedolan.github.io/jq/). Edit `C:\HD\jq\jq-win64`below to reflect your actual path and version of [jq](https://stedolan.github.io/jq/).
313
+
The PowerShell script uses [jq](https://stedolan.github.io/jq/). Edit `C:\HD\jq\jq-win64` to reflect your actual path and version of [jq](https://stedolan.github.io/jq/).
@@ -385,7 +385,7 @@ This example returns a JSON document containing the current configuration for th
385
385
386
386
At this point, the Ambari web UI indicates the Spark service needs to be restarted before the new configuration can take effect. Use the following steps to restart the service.
387
387
388
-
1. Use the following to enable maintenance mode for the Spark2 service:
388
+
1. Use the following to enable maintenance mode for the Spark 2 service:
@@ -453,7 +453,7 @@ At this point, the Ambari web UI indicates the Spark service needs to be restar
453
453
> The `href` value returned by this URI is using the internal IP address of the cluster node. To use it from outside the cluster, replace the `10.0.0.18:8080` portion with the FQDN of the cluster.
454
454
455
455
4. Verify request.
456
-
Edit the commandbelow by replacing `29` with the actual value for`id` returned from the prior step. The following commands retrieve the status of the request:
456
+
Edit the command by replacing `29` with the actual value for`id` returned from the prior step. The following commands retrieve the status of the request:
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-hadoop-script-actions-linux.md
+7-7Lines changed: 7 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -34,7 +34,7 @@ When you develop a custom script for an HDInsight cluster, there are several bes
34
34
*[Target the Apache Hadoop version](#bPS1)
35
35
*[Target the OS Version](#bps10)
36
36
*[Provide stable links to script resources](#bPS2)
37
-
*[Use pre-compiled resources](#bPS4)
37
+
*[Use precompiled resources](#bPS4)
38
38
*[Ensure that the cluster customization script is idempotent](#bPS3)
39
39
*[Ensure high availability of the cluster architecture](#bPS5)
40
40
*[Configure the custom components to use Azure Blob storage](#bPS6)
@@ -68,7 +68,7 @@ fi
68
68
69
69
### <aname="bps10"></a> Target the operating system version
70
70
71
-
HDInsight is based on the Ubuntu Linux distribution. Different versions of HDInsight rely on different versions of Ubuntu, which may change how your script behaves. For example, HDInsight 3.4 and earlier are based on Ubuntu versions that use Upstart. Versions 3.5 and greater are based on Ubuntu 16.04, which uses Systemd. Systemd and Upstart rely on different commands, so your script should be written to work with both.
71
+
HDInsight is based on the Ubuntu Linux distribution. Different versions of HDInsight rely on different versions of Ubuntu, which may change how your script behaves. For example, HDInsight 3.4 and earlier are based on Ubuntu versions that use Upstart. Versions 3.5 and greater are based on Ubuntu 16.04, which uses `Systemd`. `Systemd` and Upstart rely on different commands, so your script should be written to work with both.
72
72
73
73
Another important difference between HDInsight 3.4 and 3.5 is that `JAVA_HOME` now points to Java 8. The following code demonstrates how to determine if the script is running on Ubuntu 14 or 16:
74
74
@@ -105,7 +105,7 @@ You can find the full script that contains these snippets at https://hdiconfigac
105
105
106
106
For the version of Ubuntu that is used by HDInsight, see the [HDInsight component version](hdinsight-component-versioning.md) document.
107
107
108
-
To understand the differences between Systemd and Upstart, see [Systemd for Upstart users](https://wiki.ubuntu.com/SystemdForUpstartUsers).
108
+
To understand the differences between `Systemd` and Upstart, see [`Systemd` for Upstart users](https://wiki.ubuntu.com/SystemdForUpstartUsers).
109
109
110
110
### <aname="bPS2"></a>Provide stable links to script resources
111
111
@@ -118,15 +118,15 @@ The best practice is to download and archive everything in an Azure Storage acco
118
118
119
119
For example, the samples provided by Microsoft are stored in the `https://hdiconfigactions.blob.core.windows.net/` storage account. This location is a public, read-only container maintained by the HDInsight team.
120
120
121
-
### <aname="bPS4"></a>Use pre-compiled resources
121
+
### <aname="bPS4"></a>Use precompiled resources
122
122
123
-
To reduce the time it takes to run the script, avoid operations that compile resources from source code. For example, pre-compile resources and store them in an Azure Storage account blob in the same data center as HDInsight.
123
+
To reduce the time it takes to run the script, avoid operations that compile resources from source code. For example, precompile resources and store them in an Azure Storage account blob in the same data center as HDInsight.
124
124
125
125
### <aname="bPS3"></a>Ensure that the cluster customization script is idempotent
126
126
127
127
Scripts must be idempotent. If the script runs multiple times, it should return the cluster to the same state every time.
128
128
129
-
For example, a script that modifies configuration files shouldn't add duplicate entries if ran multiple times.
129
+
If the script runs multiple times, the script modifies configuration files shouldn't add duplicate entries.
130
130
131
131
### <aname="bPS5"></a>Ensure high availability of the cluster architecture
132
132
@@ -181,7 +181,7 @@ line 1: #!/usr/bin/env: No such file or directory
181
181
182
182
### <aname="bps9"></a> Use retry logic to recover from transient errors
183
183
184
-
When downloading files, installing packages using apt-get, or other actions that transmit data over the internet, the action may fail because of transient networking errors. For example, the remote resource you're communicating with may be in the process of failing over to a backup node.
184
+
When you download files, installing packages using apt-get, or other actions that transmit data over the internet, the action may fail because of transient networking errors. For example, the remote resource you're communicating with may be in the process of failing over to a backup node.
185
185
186
186
To make your script resilient to transient errors, you can implement retry logic. The following function demonstrates how to implement retry logic. It retries the operation three times before failing.
0 commit comments