You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/hdinsight-hadoop-linux-use-ssh-unix.md
+5-32Lines changed: 5 additions & 32 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
---
2
2
title: Use SSH with Hadoop - Azure HDInsight
3
-
description: "You can access HDInsight using Secure Shell (SSH). This document provides information on connecting to HDInsight using the ssh and scp commands from Windows, Linux, Unix, or macOS clients."
3
+
description: "You can access HDInsight using Secure Shell (SSH). This document provides information on connecting to HDInsight using the ssh commands from Windows, Linux, Unix, or macOS clients."
4
4
author: hrasheed-msft
5
5
ms.author: hrasheed
6
6
ms.reviewer: jasonh
@@ -12,7 +12,7 @@ ms.date: 02/28/2020
12
12
13
13
# Connect to HDInsight (Apache Hadoop) using SSH
14
14
15
-
Learn how to use [Secure Shell (SSH)](https://en.wikipedia.org/wiki/Secure_Shell) to securely connect to Apache Hadoop on Azure HDInsight. For information on connecting through a virtual network, see [Azure HDInsight virtual network architecture](./hdinsight-virtual-network-architecture.md) and[Plan a virtual network deployment for Azure HDInsight clusters](./hdinsight-plan-virtual-network-deployment.md).
15
+
Learn how to use [Secure Shell (SSH)](https://en.wikipedia.org/wiki/Secure_Shell) to securely connect to Apache Hadoop on Azure HDInsight. For information on connecting through a virtual network, see [Azure HDInsight virtual network architecture](./hdinsight-virtual-network-architecture.md). See also,[Plan a virtual network deployment for Azure HDInsight clusters](./hdinsight-plan-virtual-network-deployment.md).
16
16
17
17
The following table contains the address and port information needed when connecting to HDInsight using an SSH client:
18
18
@@ -84,7 +84,7 @@ You're prompted for information during the key creation process. For example, wh
84
84
| ------- | ------- |
85
85
| Azure portal | Uncheck __Use cluster login password for SSH__, and then select __Public Key__ as the SSH authentication type. Finally, select the public key file or paste the text contents of the file in the __SSH public key__ field.</br>|
86
86
| Azure PowerShell | Use the `-SshPublicKey` parameter of the [New-AzHdinsightCluster](https://docs.microsoft.com/powershell/module/az.hdinsight/new-azhdinsightcluster) cmdlet and pass the contents of the public key as a string.|
87
-
| Azure CLI | Use the `--sshPublicKey` parameter of the [az hdinsight create](https://docs.microsoft.com/cli/azure/hdinsight?view=azure-cli-latest#az-hdinsight-create) command and pass the contents of the public key as a string. |
87
+
| Azure CLI | Use the `--sshPublicKey` parameter of the [`az hdinsight create`](https://docs.microsoft.com/cli/azure/hdinsight?view=azure-cli-latest#az-hdinsight-create) command and pass the contents of the public key as a string. |
88
88
| Resource Manager Template | For an example of using SSH keys with a template, see [Deploy HDInsight on Linux with SSH key](https://azure.microsoft.com/resources/templates/101-hdinsight-linux-ssh-publickey/). The `publicKeys` element in the [azuredeploy.json](https://github.com/Azure/azure-quickstart-templates/blob/master/101-hdinsight-linux-ssh-publickey/azuredeploy.json) file is used to pass the keys to Azure when creating the cluster. |
89
89
90
90
## Authentication: Password
@@ -103,7 +103,7 @@ SSH accounts can be secured using a password. When you connect to HDInsight usin
103
103
| --------------- | ---------------- |
104
104
| Azure portal | By default, the SSH user account has the same password as the cluster login account. To use a different password, uncheck __Use cluster login password for SSH__, and then enter the password in the __SSH password__ field.</br>|
105
105
| Azure PowerShell | Use the `--SshCredential` parameter of the [New-AzHdinsightCluster](https://docs.microsoft.com/powershell/module/az.hdinsight/new-azhdinsightcluster) cmdlet and pass a `PSCredential` object that contains the SSH user account name and password. |
106
-
| Azure CLI | Use the `--ssh-password` parameter of the [az hdinsight create](https://docs.microsoft.com/cli/azure/hdinsight?view=azure-cli-latest#az-hdinsight-create) command and provide the password value. |
106
+
| Azure CLI | Use the `--ssh-password` parameter of the [`az hdinsight create`](https://docs.microsoft.com/cli/azure/hdinsight?view=azure-cli-latest#az-hdinsight-create) command and provide the password value. |
107
107
| Resource Manager Template | For an example of using a password with a template, see [Deploy HDInsight on Linux with SSH password](https://azure.microsoft.com/resources/templates/101-hdinsight-linux-ssh-password/). The `linuxOperatingSystemProfile` element in the [azuredeploy.json](https://github.com/Azure/azure-quickstart-templates/blob/master/101-hdinsight-linux-ssh-password/azuredeploy.json) file is used to pass the SSH account name and password to Azure when creating the cluster.|
108
108
109
109
### Change the SSH password
@@ -224,35 +224,8 @@ If the SSH account is secured using __SSH keys__, make sure that SSH forwarding
224
224
225
225
5. Connect to the cluster edge node or head nodes using SSH. Then use the SSH command to connect to a worker or zookeeper node. The connection is established using the forwarded key.
226
226
227
-
## Copy files
228
-
229
-
The `scp` utility can be used to copy files to and from individual nodes in the cluster. For example, the following command copies the `test.txt` directory from the local system to the primary head node:
> `scp` can only access the file system of individual nodes within the cluster. It cannot be used to access data in the HDFS-compatible storage for the cluster.
245
-
>
246
-
> Use `scp` when you need to upload a resource for use from an SSH session. For example, upload a Python script and then run the script from an SSH session.
247
-
>
248
-
> For information on directly loading data into the HDFS-compatible storage, see the following documents:
249
-
>
250
-
> * [HDInsight using Azure Storage](hdinsight-hadoop-use-blob-storage.md).
251
-
>
252
-
> * [HDInsight using Azure Data Lake Storage](hdinsight-hadoop-use-data-lake-store.md).
253
-
254
227
## Next steps
255
228
256
229
* [Use SSH tunneling with HDInsight](hdinsight-linux-ambari-ssh-tunnel.md)
257
-
* [Plan a virtual network with HDInsight](hdinsight-plan-virtual-network-deployment.md)
258
230
* [Use edge nodes in HDInsight](hdinsight-apps-use-edge-node.md#access-an-edge-node)
title: Use SCP with Apache Hadoop in Azure HDInsight
3
+
description: This document provides information on connecting to HDInsight using the ssh and scp commands.
4
+
author: hrasheed-msft
5
+
ms.author: hrasheed
6
+
ms.reviewer: jasonh
7
+
ms.service: hdinsight
8
+
ms.topic: conceptual
9
+
ms.date: 04/22/2020
10
+
---
11
+
12
+
# Use SCP with Apache Hadoop in Azure HDInsight
13
+
14
+
This article provides information on securely transferring files with your HDInsight cluster.
15
+
16
+
## Copy files
17
+
18
+
The `scp` utility can be used to copy files to and from individual nodes in the cluster. For example, the following command copies the `test.txt` directory from the local system to the primary head node:
`scp` can only access the file system of individual nodes within the cluster. It can't be used to access data in the HDFS-compatible storage for the cluster.
33
+
34
+
Use `scp` when you need to upload a resource for use from an SSH session. For example, upload a Python script and then run the script from an SSH session.
35
+
36
+
For information on directly loading data into the HDFS-compatible storage, see the following documents:
37
+
38
+
*[HDInsight using Azure Storage](hdinsight-hadoop-use-blob-storage.md).
39
+
*[HDInsight using Azure Data Lake Storage](hdinsight-hadoop-use-data-lake-store.md).
40
+
41
+
## Next steps
42
+
43
+
*[Use SSH with HDInsight](./hdinsight-hadoop-linux-use-ssh-unix.md)
44
+
*[Use edge nodes in HDInsight](hdinsight-apps-use-edge-node.md#access-an-edge-node)
0 commit comments