Skip to content

Commit c3e8b35

Browse files
authored
Merge pull request #95246 from twooley/twgen1seo
SEO work to shorten titles plus markdown cleanup
2 parents dccaa37 + 3e74778 commit c3e8b35

5 files changed

+225
-183
lines changed

articles/data-lake-store/data-lake-store-data-transfer-sql-sqoop.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Copy data between Azure Data Lake Storage Gen1 and Azure SQL database using Sqoop | Microsoft Docs
2+
title: Copy data between Data Lake Storage Gen1 and Azure SQL - Sqoop | Microsoft Docs
33
description: Use Sqoop to copy data between Azure SQL Database and Azure Data Lake Storage Gen1
44
services: data-lake-store
55
author: twooley
Lines changed: 82 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,16 @@
11
---
2-
title: Use PowerShell to get started with Azure Data Lake Storage Gen1 | Microsoft Docs
3-
description: Use Azure PowerShell to create an Azure Data Lake Storage Gen1 account and perform basic operations
4-
services: data-lake-store
5-
documentationcenter: ''
6-
author: twooley
7-
manager: mtillman
2+
title: Get started with Azure Data Lake Storage Gen1 - PowerShell | Microsoft Docs
3+
description: Use Azure PowerShell to create an Azure Data Lake Storage Gen1 account and perform basic operations.
84

5+
author: twooley
96
ms.service: data-lake-store
10-
ms.devlang: na
117
ms.topic: conceptual
128
ms.date: 06/27/2018
139
ms.author: twooley
1410

1511
---
1612
# Get started with Azure Data Lake Storage Gen1 using Azure PowerShell
13+
1714
> [!div class="op_single_selector"]
1815
> * [Portal](data-lake-store-get-started-portal.md)
1916
> * [PowerShell](data-lake-store-get-started-powershell.md)
@@ -33,92 +30,137 @@ Learn how to use Azure PowerShell to create an Azure Data Lake Storage Gen1 acco
3330
* **Azure PowerShell 1.0 or greater**. See [How to install and configure Azure PowerShell](/powershell/azure/overview).
3431

3532
## Authentication
36-
This article uses a simpler authentication approach with Data Lake Storage Gen1 where you are prompted to enter your Azure account credentials. The access level to Data Lake Storage Gen1 account and file system is then governed by the access level of the logged in user. However, there are other approaches as well to authenticate with Data Lake Storage Gen1, which are **end-user authentication** or **service-to-service authentication**. For instructions and more information on how to authenticate, see [End-user authentication](data-lake-store-end-user-authenticate-using-active-directory.md) or [Service-to-service authentication](data-lake-store-authenticate-using-active-directory.md).
33+
34+
This article uses a simpler authentication approach with Data Lake Storage Gen1 where you're prompted to enter your Azure account credentials. The access level to Data Lake Storage Gen1 account and file system is then governed by the access level of the logged in user. However, there are other approaches to authenticate with Data Lake Storage Gen1, which are end-user authentication or service-to-service authentication. For instructions and more information on how to authenticate, see [End-user authentication](data-lake-store-end-user-authenticate-using-active-directory.md) or [Service-to-service authentication](data-lake-store-authenticate-using-active-directory.md).
3735

3836
## Create a Data Lake Storage Gen1 account
37+
3938
1. From your desktop, open a new Windows PowerShell window. Enter the following snippet to log in to your Azure account, set the subscription, and register the Data Lake Storage Gen1 provider. When prompted to log in, make sure you log in as one of the subscription administrators/owner:
4039

41-
# Log in to your Azure account
42-
Connect-AzAccount
40+
```PowerShell
41+
# Log in to your Azure account
42+
Connect-AzAccount
43+
44+
# List all the subscriptions associated to your account
45+
Get-AzSubscription
4346
44-
# List all the subscriptions associated to your account
45-
Get-AzSubscription
47+
# Select a subscription
48+
Set-AzContext -SubscriptionId <subscription ID>
4649
47-
# Select a subscription
48-
Set-AzContext -SubscriptionId <subscription ID>
50+
# Register for Azure Data Lake Storage Gen1
51+
Register-AzResourceProvider -ProviderNamespace "Microsoft.DataLakeStore"
52+
```
4953
50-
# Register for Azure Data Lake Storage Gen1
51-
Register-AzResourceProvider -ProviderNamespace "Microsoft.DataLakeStore"
52-
2. A Data Lake Storage Gen1 account is associated with an Azure Resource Group. Start by creating an Azure Resource Group.
54+
1. A Data Lake Storage Gen1 account is associated with an Azure resource group. Start by creating a resource group.
5355
54-
$resourceGroupName = "<your new resource group name>"
55-
New-AzResourceGroup -Name $resourceGroupName -Location "East US 2"
56+
```PowerShell
57+
$resourceGroupName = "<your new resource group name>"
58+
New-AzResourceGroup -Name $resourceGroupName -Location "East US 2"
59+
```
5660
5761
![Create an Azure Resource Group](./media/data-lake-store-get-started-powershell/ADL.PS.CreateResourceGroup.png "Create an Azure Resource Group")
58-
3. Create a Data Lake Storage Gen1 account. The name you specify must only contain lowercase letters and numbers.
5962
60-
$dataLakeStorageGen1Name = "<your new Data Lake Storage Gen1 account name>"
61-
New-AzDataLakeStoreAccount -ResourceGroupName $resourceGroupName -Name $dataLakeStorageGen1Name -Location "East US 2"
63+
1. Create a Data Lake Storage Gen1 account. The name you specify must only contain lowercase letters and numbers.
64+
65+
```PowerShell
66+
$dataLakeStorageGen1Name = "<your new Data Lake Storage Gen1 account name>"
67+
New-AzDataLakeStoreAccount -ResourceGroupName $resourceGroupName -Name $dataLakeStorageGen1Name -Location "East US 2"
68+
```
6269
6370
![Create a Data Lake Storage Gen1 account](./media/data-lake-store-get-started-powershell/ADL.PS.CreateADLAcc.png "Create a Data Lake Storage Gen1 account")
64-
4. Verify that the account is successfully created.
6571
66-
Test-AzDataLakeStoreAccount -Name $dataLakeStorageGen1Name
72+
1. Verify that the account is successfully created.
73+
74+
```PowerShell
75+
Test-AzDataLakeStoreAccount -Name $dataLakeStorageGen1Name
76+
```
6777
6878
The output for the cmdlet should be **True**.
6979
70-
## Create directory structures in your Data Lake Storage Gen1 account
80+
## Create directory structures
81+
7182
You can create directories under your Data Lake Storage Gen1 account to manage and store data.
7283
7384
1. Specify a root directory.
7485
75-
$myrootdir = "/"
76-
2. Create a new directory called **mynewdirectory** under the specified root.
86+
```PowerShell
87+
$myrootdir = "/"
88+
```
89+
90+
1. Create a new directory called **mynewdirectory** under the specified root.
91+
92+
```PowerShell
93+
New-AzDataLakeStoreItem -Folder -AccountName $dataLakeStorageGen1Name -Path $myrootdir/mynewdirectory
94+
```
7795
78-
New-AzDataLakeStoreItem -Folder -AccountName $dataLakeStorageGen1Name -Path $myrootdir/mynewdirectory
79-
3. Verify that the new directory is successfully created.
96+
1. Verify that the new directory is successfully created.
8097
81-
Get-AzDataLakeStoreChildItem -AccountName $dataLakeStorageGen1Name -Path $myrootdir
98+
```PowerShell
99+
Get-AzDataLakeStoreChildItem -AccountName $dataLakeStorageGen1Name -Path $myrootdir
100+
```
82101
83102
It should show an output as shown in the following screenshot:
84103
85104
![Verify Directory](./media/data-lake-store-get-started-powershell/ADL.PS.Verify.Dir.Creation.png "Verify Directory")
86105
87-
## Upload data to your Data Lake Storage Gen1 account
106+
## Upload data
107+
88108
You can upload your data to Data Lake Storage Gen1 directly at the root level, or to a directory that you created within the account. The snippets in this section demonstrate how to upload some sample data to the directory (**mynewdirectory**) you created in the previous section.
89109
90110
If you are looking for some sample data to upload, you can get the **Ambulance Data** folder from the [Azure Data Lake Git Repository](https://github.com/MicrosoftBigData/usql/tree/master/Examples/Samples/Data/AmbulanceData). Download the file and store it in a local directory on your computer, such as C:\sampledata\.
91111
92-
Import-AzDataLakeStoreItem -AccountName $dataLakeStorageGen1Name -Path "C:\sampledata\vehicle1_09142014.csv" -Destination $myrootdir\mynewdirectory\vehicle1_09142014.csv
112+
```PowerShell
113+
Import-AzDataLakeStoreItem -AccountName $dataLakeStorageGen1Name `
114+
-Path "C:\sampledata\vehicle1_09142014.csv" `
115+
-Destination $myrootdir\mynewdirectory\vehicle1_09142014.csv
116+
```
93117

118+
## Rename, download, and delete data
94119

95-
## Rename, download, and delete data from your Data Lake Storage Gen1 account
96120
To rename a file, use the following command:
97121

98-
Move-AzDataLakeStoreItem -AccountName $dataLakeStorageGen1Name -Path $myrootdir\mynewdirectory\vehicle1_09142014.csv -Destination $myrootdir\mynewdirectory\vehicle1_09142014_Copy.csv
122+
```PowerShell
123+
Move-AzDataLakeStoreItem -AccountName $dataLakeStorageGen1Name `
124+
-Path $myrootdir\mynewdirectory\vehicle1_09142014.csv `
125+
-Destination $myrootdir\mynewdirectory\vehicle1_09142014_Copy.csv
126+
```
99127

100128
To download a file, use the following command:
101129

102-
Export-AzDataLakeStoreItem -AccountName $dataLakeStorageGen1Name -Path $myrootdir\mynewdirectory\vehicle1_09142014_Copy.csv -Destination "C:\sampledata\vehicle1_09142014_Copy.csv"
130+
```PowerShell
131+
Export-AzDataLakeStoreItem -AccountName $dataLakeStorageGen1Name `
132+
-Path $myrootdir\mynewdirectory\vehicle1_09142014_Copy.csv `
133+
-Destination "C:\sampledata\vehicle1_09142014_Copy.csv"
134+
```
103135

104136
To delete a file, use the following command:
105137

106-
Remove-AzDataLakeStoreItem -AccountName $dataLakeStorageGen1Name -Paths $myrootdir\mynewdirectory\vehicle1_09142014_Copy.csv
138+
```PowerShell
139+
Remove-AzDataLakeStoreItem -AccountName $dataLakeStorageGen1Name `
140+
-Paths $myrootdir\mynewdirectory\vehicle1_09142014_Copy.csv
141+
```
107142

108143
When prompted, enter **Y** to delete the item. If you have more than one file to delete, you can provide all the paths separated by comma.
109144

110-
Remove-AzDataLakeStoreItem -AccountName $dataLakeStorageGen1Name -Paths $myrootdir\mynewdirectory\vehicle1_09142014.csv, $myrootdir\mynewdirectoryvehicle1_09142014_Copy.csv
145+
```PowerShell
146+
Remove-AzDataLakeStoreItem -AccountName $dataLakeStorageGen1Name `
147+
-Paths $myrootdir\mynewdirectory\vehicle1_09142014.csv, $myrootdir\mynewdirectoryvehicle1_09142014_Copy.csv
148+
```
149+
150+
## Delete your account
111151

112-
## Delete your Data Lake Storage Gen1 account
113152
Use the following command to delete your Data Lake Storage Gen1 account.
114153

115-
Remove-AzDataLakeStoreAccount -Name $dataLakeStorageGen1Name
154+
```PowerShell
155+
Remove-AzDataLakeStoreAccount -Name $dataLakeStorageGen1Name
156+
```
116157

117158
When prompted, enter **Y** to delete the account.
118159

119160
## Next steps
161+
120162
* [Performance tuning guidance for using PowerShell with Azure Data Lake Storage Gen1](data-lake-store-performance-tuning-powershell.md)
121-
* [Use Azure Data Lake Storage Gen1 for big data requirements](data-lake-store-data-scenarios.md)
163+
* [Use Azure Data Lake Storage Gen1 for big data requirements](data-lake-store-data-scenarios.md)
122164
* [Secure data in Data Lake Storage Gen1](data-lake-store-secure-data.md)
123165
* [Use Azure Data Lake Analytics with Data Lake Storage Gen1](../data-lake-analytics/data-lake-analytics-get-started-portal.md)
124166
* [Use Azure HDInsight with Data Lake Storage Gen1](data-lake-store-hdinsight-hadoop-use-portal.md)

0 commit comments

Comments
 (0)