You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/synapse-analytics/sql/create-external-table-as-select.md
+14-15Lines changed: 14 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,7 +5,7 @@ author: vvasic-msft
5
5
ms.service: azure-synapse-analytics
6
6
ms.topic: overview
7
7
ms.subservice: sql
8
-
ms.date: 04/15/2020
8
+
ms.date: 02/20/2025
9
9
ms.author: vvasic
10
10
ms.reviewer: whhender
11
11
---
@@ -16,28 +16,31 @@ In this article, you'll learn how to store query results to storage using server
16
16
17
17
## Prerequisites
18
18
19
-
Your first step is to **create a database** where you will execute the queries. Then initialize the objects by executing [setup script](https://github.com/Azure-Samples/Synapse/blob/master/SQL/Samples/LdwSample/SampleDB.sql) on that database. This setup script will create the data sources, database scoped credentials, and external file formats that are used to read data in these samples.
19
+
Your first step is to **create a database** where you'll execute the queries. Then initialize the objects by executing [setup script](https://github.com/Azure-Samples/Synapse/blob/master/SQL/Samples/LdwSample/SampleDB.sql) on that database. This setup script will create the data sources, database scoped credentials, and external file formats that are used to read data in these samples.
20
20
21
21
Follow the instructions in this article to create data sources, database scoped credentials, and external file formats that are used to write data into the output storage.
22
22
23
23
## Create external table as select
24
24
25
-
You can use the CREATE EXTERNAL TABLE AS SELECT (CETAS) statement to store the query results to storage.
25
+
You can use the CREATE EXTERNAL TABLE AS SELECT (CETAS) statement to store the query results to storage.
26
26
27
27
> [!NOTE]
28
-
> Change the first line in the query, i.e., [mydbname], so you're using the database you created.
28
+
> Change these values in the query to reflect your environment:
29
+
> - mydbname - change it to the name of the database you created
30
+
> - storage-account-sas - the [shared access signature](/azure/ai-services/document-intelligence/authentication/create-sas-tokens#generating-sas-tokens) for a storage account where you want to write your results
31
+
> - your-storage-account-name - the name of your storage account where you want to write your results (Make sure you have a container called 'csv' or that you change the name of the container here also)
> You must modify this script and change the target location to execute it again. External tables cannot be created on the location where you already have some data.
75
+
> You must modify this script and change the target location to execute it again. External tables can't be created on the location where you already have some data.
73
76
74
77
## Use the external table
75
78
76
79
You can use the external table created through CETAS like a regular external table.
77
80
78
81
> [!NOTE]
79
-
> Change the first line in the query, i.e., [mydbname], so you're using the database you created.
82
+
> Change the first line in the query, that is, [mydbname], so you're using the database you created.
80
83
81
84
```sql
82
85
USE [mydbname];
@@ -93,14 +96,10 @@ ORDER BY
93
96
94
97
## Remarks
95
98
96
-
Once you store your results, the data in the external table cannot be modified. You cannot repeat this script because CETAS will not overwrite the underlying data created in the previous execution. Vote for the following feedback items if some of these are required in your scenarios, or propose the new ones on Azure feedback site:
97
-
-[Enable inserting new data into external table](https://feedback.azure.com/d365community/forum/9b9ba8e4-0825-ec11-b6e6-000d3a4f07b8)
98
-
-[Enable deleting data from external table](https://feedback.azure.com/d365community/idea/fb5a00c9-0a25-ec11-b6e6-000d3a4f07b8)
99
-
-[Specify partitions in CETAS](https://feedback.azure.com/d365community/idea/e28278db-0a25-ec11-b6e6-000d3a4f07b8)
100
-
-[Specify file sizes and counts](https://feedback.azure.com/d365community/idea/262048b9-0925-ec11-b6e6-000d3a4f07b8)
99
+
Once you store your results, the data in the external table can't be modified. You can't repeat this script because CETAS won't overwrite the underlying data created in the previous execution.
101
100
102
-
The only supported output types are Parquet and CSV. You can vote for the other types on [Azure feedback site](https://feedback.azure.com/d365community/forum/9b9ba8e4-0825-ec11-b6e6-000d3a4f07b8).
101
+
The only supported output types are currently Parquet and CSV.
103
102
104
-
## Next steps
103
+
## Related content
105
104
106
105
For more information on how to query different file types, see the [Query single CSV file](query-single-csv-file.md), [Query Parquet files](query-parquet-files.md), and [Query JSON files](query-json-files.md) articles.
Copy file name to clipboardExpand all lines: articles/synapse-analytics/sql/create-use-external-tables.md
+8-7Lines changed: 8 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,6 +15,7 @@ ms.reviewer: whhender, wiassaf
15
15
In this section, you'll learn how to create and use [native external tables](develop-tables-external-tables.md) in Synapse SQL pools. Native external tables have better performance when compared to external tables with `TYPE=HADOOP` in their external data source definition. This is because native external tables use native code to access external data.
16
16
17
17
External tables are useful when you want to control access to external data in Synapse SQL pool. External tables are also useful if you want to use tools, such as Power BI, in conjunction with Synapse SQL pool. External tables can access two types of storage:
18
+
18
19
- Public storage where users access public storage files.
19
20
- Protected storage where users access storage files using SAS credential, Microsoft Entra identity, or Managed Identity of Synapse workspace.
20
21
@@ -31,7 +32,7 @@ The following table lists the data formats supported:
31
32
|delta | Yes | No |
32
33
|Spark | Yes | No |
33
34
|Dataverse | Yes | No |
34
-
|Azure Cosmos DB data formats (JSON, BSON etc.) | No (Alternatively, [create views](query-cosmos-db-analytical-store.md?tabs=openrowset-credential#create-view)) | No |
35
+
|Azure Cosmos DB data formats (JSON, BSON, etc.) | No (Alternatively, [create views](query-cosmos-db-analytical-store.md?tabs=openrowset-credential#create-view)) | No |
35
36
36
37
## Prerequisites
37
38
@@ -41,7 +42,7 @@ Your first step is to create a database where the tables will be created. Before
- EXTERNAL DATA SOURCE `sqlondemanddemo` that references demo storage account protected with SAS key, and EXTERNAL DATA SOURCE `nyctlc` that references publicly available Azure storage account on location `https://azureopendatastorage.blob.core.windows.net/nyctlc/`.
@@ -83,7 +84,7 @@ You can create external tables that access data on an Azure storage account that
83
84
The following query creates an external table that reads *population.csv* file from SynapseSQL demo Azure storage account that is referenced using `sqlondemanddemo` data source and protected with database scoped credential called `sqlondemand`.
84
85
85
86
> [!NOTE]
86
-
> Change the first linein the query, i.e., [mydbname], so you're using the database you created.
87
+
> Change the first linein the query, that is, [mydbname], so you're using the database you created.
87
88
88
89
```sql
89
90
USE [mydbname];
@@ -129,7 +130,7 @@ CREATE EXTERNAL TABLE Taxi (
129
130
You can specify the pattern that the files must satisfy in order to be referenced by the external table. The pattern is required only for Parquet and CSV tables. If you're using Delta Lake format, you need to specify just a root folder, and the external table will automatically find the pattern.
130
131
131
132
> [!NOTE]
132
-
> The table is created on partitioned folder structure, but you cannot leverage some partition elimination. If you want to get better performance by skipping the files that do not satisfy some criterion (like specific year or month in this case), use [views on external data](create-use-views.md#partitioned-views).
133
+
> The table is created on partitioned folder structure, but you cannot leverage some partition elimination. If you want to get better performance by skipping the files that don't satisfy some criterion (like specific year or month in this case), use [views on external data](create-use-views.md#partitioned-views).
133
134
134
135
## External table on appendable files
135
136
@@ -153,7 +154,7 @@ WITH (
153
154
);
154
155
```
155
156
156
-
The `ALLOW_INCONSISTENT_READS` read option will disable file modification time check during the query lifecycle and read whatever is available in the files that are referenced by the external table. In appendable files, the existing content isn't updated, and only new rows are added. Therefore, the probability of wrong results is minimized compared to the updateable files. This option might enable you to read the frequently appended files without handling the errors.
157
+
The `ALLOW_INCONSISTENT_READS` read option disables file modification time check during the query lifecycle and read whatever is available in the files that are referenced by the external table. In appendable files, the existing content isn't updated, and only new rows are added. Therefore, the probability of wrong results is minimized compared to the updateable files. This option might enable you to read the frequently appended files without handling the errors.
157
158
158
159
This option is available only in the external tables created on CSV file format.
159
160
@@ -188,7 +189,7 @@ External tables can't be created on a partitioned folder. Review the other known
188
189
External tables in serverless SQL pools don't support partitioning on Delta Lake format. Use [Delta partitioned views](create-use-views.md#delta-lake-partitioned-views) instead of tables if you have partitioned Delta Lake data sets.
189
190
190
191
> [!IMPORTANT]
191
-
> Do not create external tables on partitioned Delta Lake folders even if you see that they might work in some cases. Using unsupported features like external tables on partitioned delta folders might cause issues or instability of the serverless pool. Azure support will not be able to resolve any issue if it is using tables on partitioned folders. You would be asked to transition to [Delta partitioned views](create-use-views.md#delta-lake-partitioned-views) and rewrite your code to use only the supported feature before proceeding with issue resolution.
192
+
> Don't create external tables on partitioned Delta Lake folders even if you see that they might work in some cases. Using unsupported features like external tables on partitioned delta folders might cause issues or instability of the serverless pool. Azure support won't be able to resolve any issue if it's using tables on partitioned folders. You would be asked to transition to [Delta partitioned views](create-use-views.md#delta-lake-partitioned-views) and rewrite your code to use only the supported feature before proceeding with issue resolution.
192
193
193
194
## Use an external table
194
195
@@ -197,7 +198,7 @@ You can use [external tables](develop-tables-external-tables.md) in your queries
197
198
The following query demonstrates this using the *population* external table we created in previous section. It returns country/region names with their population in 2019 in descending order.
198
199
199
200
> [!NOTE]
200
-
> Change the first line in the query, i.e., [mydbname], so you're using the database you created.
201
+
> Change the first line in the query, that is, [mydbname], so you're using the database you created.
0 commit comments