Skip to content

Commit dfbba77

Browse files
authored
Merge pull request #112379 from dagiro/freshness_c28
Freshness c28
2 parents f077b27 + aac4016 commit dfbba77

File tree

3 files changed

+51
-32
lines changed

3 files changed

+51
-32
lines changed

articles/hdinsight/TOC.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,8 @@
131131
href: hdinsight-hadoop-linux-use-ssh-unix.md
132132
- name: Use SSH tunneling
133133
href: hdinsight-linux-ambari-ssh-tunnel.md
134+
- name: Use SCP with HDInsight
135+
href: ./use-scp.md
134136
- name: Restrict access to data
135137
href: hdinsight-storage-sharedaccesssignature-permissions.md
136138
- name: Authorize users for Apache Ambari Views

articles/hdinsight/hdinsight-hadoop-linux-use-ssh-unix.md

Lines changed: 5 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: Use SSH with Hadoop - Azure HDInsight
3-
description: "You can access HDInsight using Secure Shell (SSH). This document provides information on connecting to HDInsight using the ssh and scp commands from Windows, Linux, Unix, or macOS clients."
3+
description: "You can access HDInsight using Secure Shell (SSH). This document provides information on connecting to HDInsight using the ssh commands from Windows, Linux, Unix, or macOS clients."
44
author: hrasheed-msft
55
ms.author: hrasheed
66
ms.reviewer: jasonh
@@ -12,7 +12,7 @@ ms.date: 02/28/2020
1212

1313
# Connect to HDInsight (Apache Hadoop) using SSH
1414

15-
Learn how to use [Secure Shell (SSH)](https://en.wikipedia.org/wiki/Secure_Shell) to securely connect to Apache Hadoop on Azure HDInsight. For information on connecting through a virtual network, see [Azure HDInsight virtual network architecture](./hdinsight-virtual-network-architecture.md) and [Plan a virtual network deployment for Azure HDInsight clusters](./hdinsight-plan-virtual-network-deployment.md).
15+
Learn how to use [Secure Shell (SSH)](https://en.wikipedia.org/wiki/Secure_Shell) to securely connect to Apache Hadoop on Azure HDInsight. For information on connecting through a virtual network, see [Azure HDInsight virtual network architecture](./hdinsight-virtual-network-architecture.md). See also, [Plan a virtual network deployment for Azure HDInsight clusters](./hdinsight-plan-virtual-network-deployment.md).
1616

1717
The following table contains the address and port information needed when connecting to HDInsight using an SSH client:
1818

@@ -84,7 +84,7 @@ You're prompted for information during the key creation process. For example, wh
8484
| ------- | ------- |
8585
| Azure portal | Uncheck __Use cluster login password for SSH__, and then select __Public Key__ as the SSH authentication type. Finally, select the public key file or paste the text contents of the file in the __SSH public key__ field.</br>![SSH public key dialog in HDInsight cluster creation](./media/hdinsight-hadoop-linux-use-ssh-unix/create-hdinsight-ssh-public-key.png) |
8686
| Azure PowerShell | Use the `-SshPublicKey` parameter of the [New-AzHdinsightCluster](https://docs.microsoft.com/powershell/module/az.hdinsight/new-azhdinsightcluster) cmdlet and pass the contents of the public key as a string.|
87-
| Azure CLI | Use the `--sshPublicKey` parameter of the [az hdinsight create](https://docs.microsoft.com/cli/azure/hdinsight?view=azure-cli-latest#az-hdinsight-create) command and pass the contents of the public key as a string. |
87+
| Azure CLI | Use the `--sshPublicKey` parameter of the [`az hdinsight create`](https://docs.microsoft.com/cli/azure/hdinsight?view=azure-cli-latest#az-hdinsight-create) command and pass the contents of the public key as a string. |
8888
| Resource Manager Template | For an example of using SSH keys with a template, see [Deploy HDInsight on Linux with SSH key](https://azure.microsoft.com/resources/templates/101-hdinsight-linux-ssh-publickey/). The `publicKeys` element in the [azuredeploy.json](https://github.com/Azure/azure-quickstart-templates/blob/master/101-hdinsight-linux-ssh-publickey/azuredeploy.json) file is used to pass the keys to Azure when creating the cluster. |
8989

9090
## Authentication: Password
@@ -103,7 +103,7 @@ SSH accounts can be secured using a password. When you connect to HDInsight usin
103103
| --------------- | ---------------- |
104104
| Azure portal | By default, the SSH user account has the same password as the cluster login account. To use a different password, uncheck __Use cluster login password for SSH__, and then enter the password in the __SSH password__ field.</br>![SSH password dialog in HDInsight cluster creation](./media/hdinsight-hadoop-linux-use-ssh-unix/create-hdinsight-ssh-password.png)|
105105
| Azure PowerShell | Use the `--SshCredential` parameter of the [New-AzHdinsightCluster](https://docs.microsoft.com/powershell/module/az.hdinsight/new-azhdinsightcluster) cmdlet and pass a `PSCredential` object that contains the SSH user account name and password. |
106-
| Azure CLI | Use the `--ssh-password` parameter of the [az hdinsight create](https://docs.microsoft.com/cli/azure/hdinsight?view=azure-cli-latest#az-hdinsight-create) command and provide the password value. |
106+
| Azure CLI | Use the `--ssh-password` parameter of the [`az hdinsight create`](https://docs.microsoft.com/cli/azure/hdinsight?view=azure-cli-latest#az-hdinsight-create) command and provide the password value. |
107107
| Resource Manager Template | For an example of using a password with a template, see [Deploy HDInsight on Linux with SSH password](https://azure.microsoft.com/resources/templates/101-hdinsight-linux-ssh-password/). The `linuxOperatingSystemProfile` element in the [azuredeploy.json](https://github.com/Azure/azure-quickstart-templates/blob/master/101-hdinsight-linux-ssh-password/azuredeploy.json) file is used to pass the SSH account name and password to Azure when creating the cluster.|
108108

109109
### Change the SSH password
@@ -224,35 +224,8 @@ If the SSH account is secured using __SSH keys__, make sure that SSH forwarding
224224
225225
5. Connect to the cluster edge node or head nodes using SSH. Then use the SSH command to connect to a worker or zookeeper node. The connection is established using the forwarded key.
226226
227-
## Copy files
228-
229-
The `scp` utility can be used to copy files to and from individual nodes in the cluster. For example, the following command copies the `test.txt` directory from the local system to the primary head node:
230-
231-
```bash
232-
scp test.txt [email protected]:
233-
```
234-
235-
Since no path is specified after the `:`, the file is placed in the `sshuser` home directory.
236-
237-
The following example copies the `test.txt` file from the `sshuser` home directory on the primary head node to the local system:
238-
239-
```bash
240-
scp [email protected]:test.txt .
241-
```
242-
243-
> [!IMPORTANT]
244-
> `scp` can only access the file system of individual nodes within the cluster. It cannot be used to access data in the HDFS-compatible storage for the cluster.
245-
>
246-
> Use `scp` when you need to upload a resource for use from an SSH session. For example, upload a Python script and then run the script from an SSH session.
247-
>
248-
> For information on directly loading data into the HDFS-compatible storage, see the following documents:
249-
>
250-
> * [HDInsight using Azure Storage](hdinsight-hadoop-use-blob-storage.md).
251-
>
252-
> * [HDInsight using Azure Data Lake Storage](hdinsight-hadoop-use-data-lake-store.md).
253-
254227
## Next steps
255228
256229
* [Use SSH tunneling with HDInsight](hdinsight-linux-ambari-ssh-tunnel.md)
257-
* [Plan a virtual network with HDInsight](hdinsight-plan-virtual-network-deployment.md)
258230
* [Use edge nodes in HDInsight](hdinsight-apps-use-edge-node.md#access-an-edge-node)
231+
* [Use SCP with HDInsight](./use-scp.md)

articles/hdinsight/use-scp.md

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
---
2+
title: Use SCP with Apache Hadoop in Azure HDInsight
3+
description: This document provides information on connecting to HDInsight using the ssh and scp commands.
4+
author: hrasheed-msft
5+
ms.author: hrasheed
6+
ms.reviewer: jasonh
7+
ms.service: hdinsight
8+
ms.topic: conceptual
9+
ms.date: 04/22/2020
10+
---
11+
12+
# Use SCP with Apache Hadoop in Azure HDInsight
13+
14+
This article provides information on securely transferring files with your HDInsight cluster.
15+
16+
## Copy files
17+
18+
The `scp` utility can be used to copy files to and from individual nodes in the cluster. For example, the following command copies the `test.txt` directory from the local system to the primary head node:
19+
20+
```bash
21+
scp test.txt [email protected]:
22+
```
23+
24+
Since no path is specified after the `:`, the file is placed in the `sshuser` home directory.
25+
26+
The following example copies the `test.txt` file from the `sshuser` home directory on the primary head node to the local system:
27+
28+
```bash
29+
scp [email protected]:test.txt .
30+
```
31+
32+
`scp` can only access the file system of individual nodes within the cluster. It can't be used to access data in the HDFS-compatible storage for the cluster.
33+
34+
Use `scp` when you need to upload a resource for use from an SSH session. For example, upload a Python script and then run the script from an SSH session.
35+
36+
For information on directly loading data into the HDFS-compatible storage, see the following documents:
37+
38+
* [HDInsight using Azure Storage](hdinsight-hadoop-use-blob-storage.md).
39+
* [HDInsight using Azure Data Lake Storage](hdinsight-hadoop-use-data-lake-store.md).
40+
41+
## Next steps
42+
43+
* [Use SSH with HDInsight](./hdinsight-hadoop-linux-use-ssh-unix.md)
44+
* [Use edge nodes in HDInsight](hdinsight-apps-use-edge-node.md#access-an-edge-node)

0 commit comments

Comments
 (0)