You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/storage/blobs/data-lake-storage-use-sql.md
+38-68Lines changed: 38 additions & 68 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,15 +1,14 @@
1
1
---
2
2
title: 'Tutorial: Azure Data Lake Storage Gen2, Azure Synapse'
3
3
titleSuffix: Azure Storage
4
-
description: This tutorial shows how to run SQL queries on an Azure Synapse serverless SQL endpoint to access data in an Azure Data Lake Storage Gen2 storage account.
4
+
description: This tutorial shows how to run SQL queries on an Azure Synapse serverless SQL endpoint to access data in an Azure Data Lake Storage Gen2 enabled storage account.
5
5
author: jovanpop-msft
6
6
7
7
ms.subservice: data-lake-storage-gen2
8
8
ms.service: storage
9
9
ms.topic: tutorial
10
-
ms.date: 11/22/2021
11
-
ms.author: jovanpop
12
-
ms.reviewer: jrasnic
10
+
ms.date: 03/07/2022
11
+
ms.author: normesta
13
12
ms.custom: devx-track-sql
14
13
#Customer intent: As an data engineer, I want to connect my data in Azure Storage so that I can easily run analytics on it.
15
14
---
@@ -30,114 +29,85 @@ If you don't have an Azure subscription, create a [free account](https://azure.m
30
29
31
30
## Prerequisites
32
31
33
-
- Create an Azure Data Lake Storage Gen2 account.
32
+
- Create a storage account that has a hierarchical namespace (Azure Data Lake Storage Gen2)
34
33
35
34
See [Create a storage account to use with Azure Data Lake Storage Gen2](create-data-lake-storage-account.md).
36
35
37
36
- Make sure that your user account has the [Storage Blob Data Contributor role](assign-azure-role-data-access.md) assigned to it.
38
37
39
-
- Install AzCopy v10. See [Transfer data with AzCopy v10](../common/storage-use-azcopy-v10.md?toc=/azure/storage/blobs/toc.json)
40
-
41
-
There's a couple of specific things that you'll have to do as you perform the steps in that article.
42
-
43
38
> [!IMPORTANT]
44
-
> Make sure to assign the role in the scope of the Data Lake Storage Gen2 storage account. You can assign a role to the parent resource group or subscription, but you'll receive permissions-related errors until those role assignments propagate to the storage account.
39
+
> Make sure to assign the role in the scope of the storage account. You can assign a role to the parent resource group or subscription, but you'll receive permissions-related errors until those role assignments propagate to the storage account.
45
40
46
41
### Download the flight data
47
42
48
43
This tutorial uses flight data from the Bureau of Transportation Statistics to demonstrate how to perform an ETL operation. You must download this data to complete the tutorial.
49
44
50
-
1. Go to [Research and Innovative Technology Administration, Bureau of Transportation Statistics](https://www.transtats.bts.gov/DL_SelectFields.asp?gnoyr_VQ=FGJ).
51
-
52
-
2. Select the **Prezipped File** check box to select all data fields.
45
+
1. Download the [On_Time_Reporting_Carrier_On_Time_Performance_1987_present_2016_1.zip](https://github.com/Azure-Samples/AzureStorageSnippets/blob/master/blobs/tutorials/On_Time_Reporting_Carrier_On_Time_Performance_1987_present_2016_1.zip) file. This file contains the flight data.
53
46
54
-
3. Select the **Download** button and save the results to your computer.
55
-
56
-
4. Unzip the contents of the zipped file and make a note of the file name and the path of the file. You need this information in a later step.
47
+
2. Unzip the contents of the zipped file and make a note of the file name and the path of the file. You need this information in a later step.
57
48
58
49
### Copy source data into the storage account
59
50
60
-
Use AzCopy to copy data from your *.csv* file into your Data Lake Storage Gen2 account.
61
-
62
-
1. Open a command prompt window, and enter the following command to log into your storage account.
63
-
64
-
```bash
65
-
azcopy login
66
-
```
67
-
68
-
Follow the instructions that appear in the command prompt window to authenticate your user account.
69
-
70
-
2. To copy data from the *.csv* account, enter the following command.
51
+
1. Navigate to your new storage account in the Azure portal.
- Replace the `<csv-folder-path>` placeholder value with the path to the *.csv* file.
53
+
2. Select **Storage browser**->**Blob containers**->**Add container** and create a new container named **data**.
77
54
78
-
- Replace the `<storage-account-name>` placeholder value with the name of your storage account.
55
+
> [!div class="mx-imgBorder"]
56
+
> 
79
57
80
-
- Replace the `<container-name>` placeholder with the name of a container in your storage account.
58
+
6. In storage browser, upload this file to the **data** folder.
81
59
82
60
## Create an Azure Synapse workspace
83
61
84
-
In this section, you create an Azure Workspace.
85
-
86
-
1. Select the **Deploy to Azure** button. The template will open in the Azure portal.
62
+
1. Create an Azure Synapse workspace. See [Create a Synapse workspace in the Azure portal](../../synapse-analytics/get-started-create-workspace.md#create-a-synapse-workspace-in-the-azure-portal).
87
63
88
-
[](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A//raw.githubusercontent.com/Azure-Samples/Synapse/master/Manage/DeployWorkspace/azuredeploy.json)
64
+
As you create the workspace, use these values:
89
65
90
-
2. Enter or update the following values:
91
-
92
-
-**Subscription**: Select the Azure subscription where you have the Azure storage account
93
-
-**Resource group**: Select the resource group where you placed your Azure Data Lake storage.
94
-
-**Region**: Select the region where you placed your Azure Data Lake storage (for example, **Central US**).
66
+
-**Subscription**: Select the Azure subscription associated with your storage account.
67
+
-**Resource group**: Select the resource group where you placed your storage account.
68
+
-**Region**: Select the region of the storage account (for example, `Central US`).
95
69
-**Name**: Enter a name for your Synapse workspace.
96
70
-**SQL Administrator login**: Enter the administrator username for the SQL Server.
97
71
-**SQL Administrator password**: Enter the administrator password for the SQL Server.
98
72
-**Tag Values**: Accept the default.
99
73
-**Review and Create**: Select.
100
74
-**Create**: Select.
101
75
102
-
When the deployment finishes, you will see Azure Synapse Analytics workspace in the list of the deployed resources. You can follow the link to see more details about the workspace and [find your Synapse SQL endpoint name](#find-your-synapse-sql-endpoint-name).
76
+
When the deployment finishes, you will see Azure Synapse Analytics workspace in the list of the deployed resources.
77
+
78
+
## (optional) Find your Synapse SQL endpoint name
103
79
104
-
## Find your Synapse SQL endpoint name
80
+
The server name for the serverless SQL pool in the following example is: `showdemoweu-ondemand.sql.azuresynapse.net`. This endpoint enables you to connect with any tool that can run T-SQL queries on SQL server or Azure SQL database (For example: [SQL Server Management Studio](../../synapse-analytics/sql/get-started-ssms.md),
81
+
[Azure Data Studio](../../synapse-analytics/sql/get-started-azure-data-studio.md), or [Power BI](../../synapse-analytics/sql/get-started-power-bi-professional.md)).
105
82
106
-
The server name for the serverless SQL pool in the following example is: `showdemoweu-ondemand.sql.azuresynapse.net`. To find the fully qualified server name:
83
+
To find the fully qualified server name:
107
84
108
85
1. Select on the workspace you want to connect to.
109
86
2. Go to overview.
110
87
3. Locate the full server name.
111
88
112
89

113
90
114
-
### Connect to Synapse SQL endpoint
91
+
##Use Synapse Studio to explore data
115
92
116
-
Synapse SQL endpoint enables you to connect with any tool that can run T-SQL queries on SQL server or Azure SQL database.
117
-
The examples are [SQL Server Management Studio](../../synapse-analytics/sql/get-started-ssms.md),
118
-
[Azure Data Studio](../../synapse-analytics/sql/get-started-azure-data-studio.md), or [Power BI](../../synapse-analytics/sql/get-started-power-bi-professional.md),
93
+
In this tutorial, you'll use Synapse Studio to query data from the CSV file that you uploaded to the storage account.
119
94
120
-
Use a tool that you prefer to use to connect to SQL endpoint, put the serverless SQL serverless endpoint name, and connect with Azure AD authentication to connect.
95
+
1. Open Synapse Studio. See [Open Synapse Studio](../../synapse-analytics/quickstart-create-workspace.md#open-synapse-studio)
121
96
122
-
> [!IMPORTANT]
123
-
> Do not use SQL authentication with username nad password because this will require additional steps to enable SQL login to access your Azure storage account.
124
-
125
-
## Explore data
97
+
2. Create a SQL script and run this query to view the contents of the file:
126
98
127
-
Create a new SQL query using the tool that you used to connect to your Synapse endpoint, put the following query, and set the path in
When you execute the query, you will see the content of the file.
110
+
For information about how to create a SQL script in Synapse Studio, see [Synapse Studio SQL scripts in Azure Synapse Analytics](../../synapse-analytics/sql/author-sql-script.md)
0 commit comments