You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/synapse-analytics/sql/develop-tables-external-tables.md
+20-13Lines changed: 20 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,23 +15,25 @@ ms.reviewer: jrasnick
15
15
16
16
An external table points to data located in Hadoop, Azure Storage blob, or Azure Data Lake Storage. External tables are used to read data from files or write data to files in Azure Storage. With Synapse SQL, you can use external tables to read and write data to SQL pool or SQL on-demand (preview).
17
17
18
-
## External tables in Synapse SQL
18
+
## External tables in Synapse SQL pool and on-demand
19
19
20
-
### [SQL pool](#tab/sql-pool)
20
+
### [SQL pool](#tab/sql-pool)
21
21
22
22
In SQL pool, you can use an external table to:
23
23
24
24
- Query Azure Blob Storage and Azure Data Lake Gen2 with Transact-SQL statements.
25
25
- Import and store data from Azure Blob Storage and Azure Data Lake Storage into SQL pool.
26
26
27
-
When used in conjunction with the [CREATE TABLE AS SELECT](../sql-data-warehouse/sql-data-warehouse-develop-ctas.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json) statement, selecting from an external table imports data into a table within the SQL pool. In additional to the [COPY statement](/sql/t-sql/statements/copy-into-transact-sql?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json&view=azure-sqldw-latest), external tables are useful for loading data. For a loading tutorial, see [Use PolyBase to load data from Azure Blob Storage](../sql-data-warehouse/load-data-from-azure-blob-storage-using-polybase.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json).
27
+
When used in conjunction with the [CREATE TABLE AS SELECT](../sql-data-warehouse/sql-data-warehouse-develop-ctas.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json) statement, selecting from an external table imports data into a table within the SQL pool. In addition to the [COPY statement](/sql/t-sql/statements/copy-into-transact-sql?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json&view=azure-sqldw-latest), external tables are useful for loading data.
28
28
29
-
### [SQL on-demand](#tab/sql-ondemand)
29
+
For a loading tutorial, see [Use PolyBase to load data from Azure Blob Storage](../sql-data-warehouse/load-data-from-azure-blob-storage-using-polybase.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json).
30
+
31
+
### [SQL on-demand](#tab/sql-on-demand)
30
32
31
33
For SQL on-demand, you'll use an external table to:
32
34
33
35
- Query data in Azure Blob Storage or Azure Data Lake Storage with Transact-SQL statements
34
-
- Store SQL on-demand query results to files in Azure Blob Storage or Azure Data Lake Storage using [CETAS](develop-tables-cetas.md).
36
+
- Store SQL on-demand query results to files in Azure Blob Storage or Azure Data Lake Storage using [CETAS](develop-tables-cetas.md)
35
37
36
38
You can create external tables using SQL on-demand via the following steps:
37
39
@@ -45,8 +47,8 @@ You can create external tables using SQL on-demand via the following steps:
45
47
46
48
User must have `SELECT` permission on external table to read the data.
47
49
External table access underlying Azure storage using the database scoped credential defined in data source using the following rules:
48
-
- Data source without credential enable external tables to access publicly available files on Azure storage.
49
-
- Data source can have credential that enables external tables to access only the files on Azure storage using SAS token or workspace Managed Identity - see [examples here](develop-storage-files-storage-access-control.md#examples).
50
+
- Data source without credential enables external tables to access publicly available files on Azure storage.
51
+
- Data source can have credential that enables external tables to access only the files on Azure storage using SAS token or workspace Managed Identity - For examples, see [the Develop storage files storage access control](develop-storage-files-storage-access-control.md#examples) article.
50
52
51
53
> [!IMPORTANT]
52
54
> In SQL pool, datasource without creadential enables Azure AD user to access storage files using their Azure AD identity. In SQL on-demand, you need to create data source with database-scoped credential that has `IDENTITY='User Identity'` property - see [examples here](develop-storage-files-storage-access-control.md#examples).
@@ -69,7 +71,7 @@ WITH
69
71
[;]
70
72
```
71
73
72
-
#### [SQL on-demand](#tab/sql-ondemand)
74
+
#### [SQL on-demand](#tab/sql-on-demand)
73
75
74
76
```syntaxsql
75
77
CREATE EXTERNAL DATA SOURCE <data_source_name>
@@ -79,11 +81,14 @@ WITH
79
81
)
80
82
[;]
81
83
```
84
+
82
85
---
83
86
84
87
### Arguments for CREATE EXTERNAL DATA SOURCE
85
88
86
-
data_source_name -Specifies the user-defined name for the data source. The name must be unique within the database.
89
+
data_source_name
90
+
91
+
Specifies the user-defined name for the data source. The name must be unique within the database.
87
92
88
93
#### Location
89
94
LOCATION = `'<prefix>://<path>'` - Provides the connectivity protocol and path to the external data source. The path can include a container in the form of `'<prefix>://<path>/container'`, and a folder in the form of `'<prefix>://<path>/container/folder'`.
@@ -95,12 +100,14 @@ LOCATION = `'<prefix>://<path>'` - Provides the connectivity protocol and path
95
100
| Azure Data Lake Store Gen 2 |`abfs[s]`|`<container>@<storage_account>.dfs.core.windows.net`|
96
101
97
102
#### Credential
98
-
CREDENTIAL = `<database scoped credential>` is optional credential that will be used to authenticate on Azure storage. External data source without credential can access public storage account. External data sources without credential in SQL pool can also use callers Azure AD identity to access files on storage. External data source with credential use identity specified in credential to access files.
103
+
CREDENTIAL = `<database scoped credential>` is optional credential that will be used to authenticate on Azure storage. External data source without credential can access public storage account.
104
+
105
+
External data sources without credential in SQL pool can also use callers Azure AD identity to access files on storage. External data source with credential use identity specified in credential to access files.
99
106
- In SQL pool, database scoped credential can specify custom application identity, workspace Managed Identity, or SAK key.
100
107
- In SQL on-demand, database scoped credential can specify caller's Azure AD identity, workspace Managed Identity, or SAS key.
101
108
102
109
#### TYPE
103
-
TYPE = `HADOOP` is mandatory option in SQL pool and specify that Polybase technology is used to access underlying files. This parameter cannot be used in SQL on-demand service that uses built-in native reader.
110
+
TYPE = `HADOOP` is mandatory option in SQL pool and specify that Polybase technology is used to access underlying files. This parameter can't be used in SQL on-demand service that uses built-in native reader.
104
111
105
112
### Example for CREATE EXTERNAL DATA SOURCE
106
113
@@ -118,7 +125,7 @@ WITH
118
125
) ;
119
126
```
120
127
121
-
#### [SQL on-demand](#tab/sql-ondemand)
128
+
#### [SQL on-demand](#tab/sql-on-demand)
122
129
123
130
The following example creates an external data source for Azure Data Lake Gen2 that can be accessed using SAS credential:
124
131
@@ -344,7 +351,7 @@ Using Data Lake exploration capabilities you can now create and query an externa
344
351
345
352
- You must have at least [permissions to create](/sql/t-sql/statements/create-external-table-transact-sql?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json&view=azure-sqldw-latest#permissions-2) and query external tables on the SQL pool or SQL OD
346
353
347
-
- The linked service associated with the ADLS Gen2 Account **must have access to the file**. For example, if the linked service authentication mechanism is Managed Identity, the workspace managed identity must have at least Storage blob reader permission on the storage account
354
+
- The linked service associated with the ADLS Gen2 Account **must have access to the file**. For example, if the linked service authentication mechanism is Managed Identity, the workspace Managed Identity must have at least Storage blob reader permission on the storage account
348
355
349
356
From the Data panel, select the file that you would like to create the external table from:
0 commit comments