You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/synapse-analytics/sql/tutorial-external-tables-using-managed-identity.md
+14-16Lines changed: 14 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
---
2
-
title: 'Tutorial: create external tables or ingest data from on ADLS Gen2 using a managed identity'
2
+
title: 'Tutorial: Loading external data using a managed identity'
3
3
description: This tutorial shows how to connect to external data for queries or ingestion using a managed identity.
4
4
author: periclesrocha
5
5
ms.service: azure-synapse-analytics
@@ -11,15 +11,13 @@ ms.author: periclesrocha
11
11
ms.reviewer: WilliamDAssafMSFT
12
12
---
13
13
14
-
# Tutorial: create external tables or ingest data from on ADLS Gen2 using a managed identity
14
+
# Tutorial: Loading external data using a managed identity
15
15
16
-
Applies to: Azure Synapse Analytics
17
-
18
-
This article explains how to create external tables or ingest data from Azure Data Lake Storage Gen2 accounts using a managed identity.
16
+
This article explains how to create external tables or ingest data from Azure Data Lake Storage (ADLS) Gen2 accounts using a managed identity.
19
17
20
18
## Prerequisites:
21
19
22
-
This tutorial requires the following resources to be in place:
20
+
The following resources are required to complete this tutorial:
23
21
24
22
* An Azure Data Lake Storage Gen2 (ADLS Gen2) account
25
23
* An Azure Synapse Analytics workspace and a dedicated SQL Pool
@@ -30,14 +28,14 @@ Each Azure Synapse Analytics workspace automatically creates a managed identity
30
28
31
29
To enable your managed identity to access data on ADLS Gen2 accounts, you need to give your identity access to the source account. To grant proper permissions, follow these steps:
32
30
33
-
1. In the Azure Portal, find your storage account.
31
+
1. In the Azure portal, find your storage account.
34
32
2. Select **Data storage -> Containers**, and navigate to the folder where the source data the external table needs access to is.
35
33
3. Select **Access control (IAM)**.
36
34
4. Select **Add -> Add role assignment**.
37
35
5. In the list of job function roles, select **Storage Blob Data Contributor** and select **Next**.
38
36
6. In the Add role assignment page, select **+ Select members**. The Select members pane opens in the right-hand corner.
39
-
7. Type the name of your workspace identity until it is displayed. The workspace identity is the same as your workspace name. Pick your workspace identity and chose **Select**.
40
-
8.Back to the Add role assignment page, make sure the list of Members include your workspace identity. Once verified, select **Review + assign**.
37
+
7. Type the name of your workspace identity. The workspace identity is the same as your workspace name. When displayed, pick your workspace identity and chose **Select**.
38
+
8.In the **Add role assignment** page, make sure the list of Members include your desired Entra ID account. Once verified, select **Review + assign**.
41
39
9. In the confirmation page, review the changes and select **Review + assign**.
42
40
43
41
Your workspace identity is now a member of the Storage Blob Data Contributor role and has access to the source folder.
@@ -46,7 +44,7 @@ Note: these steps also apply to secure ADLS Gen2 accounts that are configured to
46
44
47
45
## Ingest data using COPY INTO
48
46
49
-
The COPY INTO statement provides flexible, high-throughput data ingestion into your tables, and is the primary strategy to ingest data into your dedicated SQL Pool tables. It allows users to ingest data from external locations without having to create any of the additional database objects that are required for external tables.
47
+
The COPY INTO statement provides flexible, high-throughput data ingestion into your tables, and is the primary strategy to ingest data into your dedicated SQL Pool tables. It allows users to ingest data from external locations without having to create any of the extra database objects that are required for external tables.
50
48
51
49
To run the COPY INTO statement using a workspace managed identity for authentication, use the following command:
52
50
@@ -62,7 +60,7 @@ WITH
62
60
63
61
Where:
64
62
65
-
*\<TableName> is the name of the table you will ingest data into
63
+
*\<TableName> is the name of the table you'll ingest data into
66
64
*\<AccountName> is your ADLS Gen2 account name
67
65
*\<Container> is the name of the container within your storage account where the source data is stored
68
66
*\<Folder> is the folder (or path with subfolders) where the source data is stored within your container. You can also provide a file name if pointing directly to a single file.
@@ -86,11 +84,11 @@ External tables require the following objects to be created:
86
84
4. An external file format that defines the format of the source files.
87
85
5. An external table definition that is used for queries.
88
86
89
-
To follow these steps, you will need to use the SQL editor in the Azure Synapse Workspace, or your preferred SQL client connected to your dedicated SQL Pool. Let’s look at these steps in detail.
87
+
To follow these steps, you'll need to use the SQL editor in the Azure Synapse Workspace, or your preferred SQL client connected to your dedicated SQL Pool. Let’s look at these steps in detail.
90
88
91
89
#### Create the database master key
92
90
93
-
The database master key is a symmetric key used to protect the private keys of certificates and asymmetric keys that are present in the database and secrets in database scoped credentials. If there is already a master key in the database, you do not need to create a new one.
91
+
The database master key is a symmetric key used to protect the private keys of certificates and asymmetric keys that are present in the database and secrets in database scoped credentials. If there's already a master key in the database, you don't need to create a new one.
94
92
95
93
To create a master key, use the following command:
96
94
@@ -169,11 +167,11 @@ Where:
169
167
170
168
* \<FileFormatName> is the name you want to use for your external file format
171
169
172
-
In the example above, adjust parameters such as FIELD\_TERMINATOR, STRING\_DELIMITER, FIRST\_ROW and others as needed in accordance with your source data. For more formatting options and to learn more about EXTERNAL FILE FORMAT, visit <https://learn.microsoft.com/en-us/sql/t-sql/statements/create-external-file-format-transact-sql?view=azure-sqldw-latest&tabs=delimited>.
170
+
In this example, adjust parameters such as FIELD_TERMINATOR, STRING_DELIMITER, FIRST_ROW and others as needed in accordance with your source data. For more formatting options and to learn more about EXTERNAL FILE FORMAT, visit <https://learn.microsoft.com/en-us/sql/t-sql/statements/create-external-file-format-transact-sql?view=azure-sqldw-latest&tabs=delimited>.
173
171
174
172
#### Create the external table
175
173
176
-
Now that we’ve created all the necessary objects that hold the metadata to securely access external data, it is time to create the external table. To create the external table, use the following command:
174
+
Now that all the necessary objects that hold the metadata to securely access external data are created, it's time to create the external table. To create the external table, use the following command:
177
175
178
176
```sql
179
177
-- Adjust the table name and columns to your desired name and external table schema
@@ -199,7 +197,7 @@ Where:
199
197
200
198
Make sure to adjust the table name and schema to the desired name and the schema of the data in your source files.
201
199
202
-
At this point, all the metadata required to access the external table has been created. To test your external table, use a simple query such as the one below:
200
+
At this point, all the metadata required to access the external table are created. To test your external table, use a query such as the following one to validate your work:
Copy file name to clipboardExpand all lines: articles/synapse-analytics/sql/tutorial-load-data-using-entra-id.md
+27-4Lines changed: 27 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,9 +11,7 @@ ms.author: periclesrocha
11
11
ms.reviewer: WilliamDAssafMSFT
12
12
---
13
13
14
-
# Tutorial: load data using Entra ID
15
-
16
-
Applies to: Azure Synapse Analytics
14
+
# Tutorial: Loading external data using Entra ID
17
15
18
16
This article explains how to create external tables using Entra ID passthrough.
19
17
@@ -41,7 +39,32 @@ To enable access to data on Azure Data Lake Storage (ADLS) Gen2 accounts, you ne
41
39
42
40
The Entra ID account or group is now a member of the Storage Blob Data Reader role and has access to the source folder.
43
41
44
-
# Create the required database objects
42
+
## Ingest data using COPY INTO
43
+
44
+
The COPY INTO statement provides flexible, high-throughput data ingestion into your tables, and is the primary strategy to ingest data into your dedicated SQL Pool tables. It allows users to ingest data from external locations without having to create any of the extra database objects that are required for external tables.
45
+
46
+
The COPY INTO statement uses the CREDENTIAL argument to specify the authentication mechanism used to connect to the source account. However, when authenticating using Microsoft Entra ID or to a public storage account, CREDENTIAL doesn't need to be specified. Therefore, to run the COPY INTO statement using a workspace managed identity for authentication, use the following command:
*\<TableName> is the name of the table to ingest data into
60
+
*\<AccountName> is your ADLS Gen2 account name
61
+
*\<Container> is the name of the container within your storage account where the source data is stored
62
+
*\<Folder> is the folder (or path with subfolders) where the source data is stored within your container. You can also provide a file name if pointing directly to a single file.
63
+
*\<CopyIntoOptions> is the list of any other options you wish to provide to the COPY INTO statement.
64
+
65
+
To learn more and explore the full syntax of COPY INTO, refer to <https://learn.microsoft.com/en-us/sql/t-sql/statements/copy-into-transact-sql?view=azure-sqldw-latest>.
66
+
67
+
## Create the required database objects
45
68
46
69
External tables require the following objects to be created:
0 commit comments