Skip to content

Commit d3ec254

Browse files
Merge pull request #295058 from whhender/resolving-broken-scripting
Resolving script issues and old keys
2 parents 7f7f8ed + 204496f commit d3ec254

File tree

3 files changed

+24
-24
lines changed

3 files changed

+24
-24
lines changed

articles/synapse-analytics/sql/create-external-table-as-select.md

Lines changed: 14 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ author: vvasic-msft
55
ms.service: azure-synapse-analytics
66
ms.topic: overview
77
ms.subservice: sql
8-
ms.date: 04/15/2020
8+
ms.date: 02/20/2025
99
ms.author: vvasic
1010
ms.reviewer: whhender
1111
---
@@ -16,28 +16,31 @@ In this article, you'll learn how to store query results to storage using server
1616

1717
## Prerequisites
1818

19-
Your first step is to **create a database** where you will execute the queries. Then initialize the objects by executing [setup script](https://github.com/Azure-Samples/Synapse/blob/master/SQL/Samples/LdwSample/SampleDB.sql) on that database. This setup script will create the data sources, database scoped credentials, and external file formats that are used to read data in these samples.
19+
Your first step is to **create a database** where you'll execute the queries. Then initialize the objects by executing [setup script](https://github.com/Azure-Samples/Synapse/blob/master/SQL/Samples/LdwSample/SampleDB.sql) on that database. This setup script will create the data sources, database scoped credentials, and external file formats that are used to read data in these samples.
2020

2121
Follow the instructions in this article to create data sources, database scoped credentials, and external file formats that are used to write data into the output storage.
2222

2323
## Create external table as select
2424

25-
You can use the CREATE EXTERNAL TABLE AS SELECT (CETAS) statement to store the query results to storage.
25+
You can use the CREATE EXTERNAL TABLE AS SELECT (CETAS) statement to store the query results to storage.
2626

2727
> [!NOTE]
28-
> Change the first line in the query, i.e., [mydbname], so you're using the database you created.
28+
> Change these values in the query to reflect your environment:
29+
> - mydbname - change it to the name of the database you created
30+
> - storage-account-sas - the [shared access signature](/azure/ai-services/document-intelligence/authentication/create-sas-tokens#generating-sas-tokens) for a storage account where you want to write your results
31+
> - your-storage-account-name - the name of your storage account where you want to write your results (Make sure you have a container called 'csv' or that you change the name of the container here also)
2932
3033
```sql
3134
USE [mydbname];
3235
GO
3336

3437
CREATE DATABASE SCOPED CREDENTIAL [SasTokenWrite]
3538
WITH IDENTITY = 'SHARED ACCESS SIGNATURE',
36-
SECRET = 'sv=2018-03-28&ss=bfqt&srt=sco&sp=rwdlacup&se=2019-04-18T20:42:12Z&st=2019-04-18T12:42:12Z&spr=https&sig=lQHczNvrk1KoYLCpFdSsMANd0ef9BrIPBNJ3VYEIq78%3D';
39+
SECRET = 'storage-account-sas';
3740
GO
3841

3942
CREATE EXTERNAL DATA SOURCE [MyDataSource] WITH (
40-
LOCATION = 'https://<storage account name>.blob.core.windows.net/csv', CREDENTIAL = [SasTokenWrite]
43+
LOCATION = 'https://your-storage-account-name.blob.core.windows.net/csv', CREDENTIAL = [SasTokenWrite]
4144
);
4245
GO
4346

@@ -69,14 +72,14 @@ FROM
6972
```
7073

7174
> [!NOTE]
72-
> You must modify this script and change the target location to execute it again. External tables cannot be created on the location where you already have some data.
75+
> You must modify this script and change the target location to execute it again. External tables can't be created on the location where you already have some data.
7376
7477
## Use the external table
7578

7679
You can use the external table created through CETAS like a regular external table.
7780

7881
> [!NOTE]
79-
> Change the first line in the query, i.e., [mydbname], so you're using the database you created.
82+
> Change the first line in the query, that is, [mydbname], so you're using the database you created.
8083
8184
```sql
8285
USE [mydbname];
@@ -93,14 +96,10 @@ ORDER BY
9396

9497
## Remarks
9598

96-
Once you store your results, the data in the external table cannot be modified. You cannot repeat this script because CETAS will not overwrite the underlying data created in the previous execution. Vote for the following feedback items if some of these are required in your scenarios, or propose the new ones on Azure feedback site:
97-
- [Enable inserting new data into external table](https://feedback.azure.com/d365community/forum/9b9ba8e4-0825-ec11-b6e6-000d3a4f07b8)
98-
- [Enable deleting data from external table](https://feedback.azure.com/d365community/idea/fb5a00c9-0a25-ec11-b6e6-000d3a4f07b8)
99-
- [Specify partitions in CETAS](https://feedback.azure.com/d365community/idea/e28278db-0a25-ec11-b6e6-000d3a4f07b8)
100-
- [Specify file sizes and counts](https://feedback.azure.com/d365community/idea/262048b9-0925-ec11-b6e6-000d3a4f07b8)
99+
Once you store your results, the data in the external table can't be modified. You can't repeat this script because CETAS won't overwrite the underlying data created in the previous execution.
101100

102-
The only supported output types are Parquet and CSV. You can vote for the other types on [Azure feedback site](https://feedback.azure.com/d365community/forum/9b9ba8e4-0825-ec11-b6e6-000d3a4f07b8).
101+
The only supported output types are currently Parquet and CSV.
103102

104-
## Next steps
103+
## Related content
105104

106105
For more information on how to query different file types, see the [Query single CSV file](query-single-csv-file.md), [Query Parquet files](query-parquet-files.md), and [Query JSON files](query-json-files.md) articles.

articles/synapse-analytics/sql/create-use-external-tables.md

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ ms.reviewer: whhender, wiassaf
1515
In this section, you'll learn how to create and use [native external tables](develop-tables-external-tables.md) in Synapse SQL pools. Native external tables have better performance when compared to external tables with `TYPE=HADOOP` in their external data source definition. This is because native external tables use native code to access external data.
1616

1717
External tables are useful when you want to control access to external data in Synapse SQL pool. External tables are also useful if you want to use tools, such as Power BI, in conjunction with Synapse SQL pool. External tables can access two types of storage:
18+
1819
- Public storage where users access public storage files.
1920
- Protected storage where users access storage files using SAS credential, Microsoft Entra identity, or Managed Identity of Synapse workspace.
2021

@@ -31,7 +32,7 @@ The following table lists the data formats supported:
3132
|delta | Yes | No |
3233
|Spark | Yes | No |
3334
|Dataverse | Yes | No |
34-
|Azure Cosmos DB data formats (JSON, BSON etc.) | No (Alternatively, [create views](query-cosmos-db-analytical-store.md?tabs=openrowset-credential#create-view)) | No |
35+
|Azure Cosmos DB data formats (JSON, BSON, etc.) | No (Alternatively, [create views](query-cosmos-db-analytical-store.md?tabs=openrowset-credential#create-view)) | No |
3536

3637
## Prerequisites
3738

@@ -41,7 +42,7 @@ Your first step is to create a database where the tables will be created. Before
4142
```sql
4243
CREATE DATABASE SCOPED CREDENTIAL [sqlondemand]
4344
WITH IDENTITY='SHARED ACCESS SIGNATURE',
44-
SECRET = 'sv=2018-03-28&ss=bf&srt=sco&sp=rl&st=2019-10-14T12%3A10%3A25Z&se=2061-12-31T12%3A10%3A00Z&sig=KlSU2ullCscyTS0An0nozEpo4tO5JAgGBvw%2FJX2lguw%3D'
45+
SECRET = 'sv=2022-11-02&ss=b&srt=co&sp=rl&se=2042-11-26T17:40:55Z&st=2024-11-24T09:40:55Z&spr=https&sig=DKZDuSeZhuCWP9IytWLQwu9shcI5pTJ%2Fw5Crw6fD%2BC8%3D'
4546
```
4647

4748
- EXTERNAL DATA SOURCE `sqlondemanddemo` that references demo storage account protected with SAS key, and EXTERNAL DATA SOURCE `nyctlc` that references publicly available Azure storage account on location `https://azureopendatastorage.blob.core.windows.net/nyctlc/`.
@@ -83,7 +84,7 @@ You can create external tables that access data on an Azure storage account that
8384
The following query creates an external table that reads *population.csv* file from SynapseSQL demo Azure storage account that is referenced using `sqlondemanddemo` data source and protected with database scoped credential called `sqlondemand`.
8485

8586
> [!NOTE]
86-
> Change the first line in the query, i.e., [mydbname], so you're using the database you created.
87+
> Change the first line in the query, that is, [mydbname], so you're using the database you created.
8788
8889
```sql
8990
USE [mydbname];
@@ -129,7 +130,7 @@ CREATE EXTERNAL TABLE Taxi (
129130
You can specify the pattern that the files must satisfy in order to be referenced by the external table. The pattern is required only for Parquet and CSV tables. If you're using Delta Lake format, you need to specify just a root folder, and the external table will automatically find the pattern.
130131
131132
> [!NOTE]
132-
> The table is created on partitioned folder structure, but you cannot leverage some partition elimination. If you want to get better performance by skipping the files that do not satisfy some criterion (like specific year or month in this case), use [views on external data](create-use-views.md#partitioned-views).
133+
> The table is created on partitioned folder structure, but you cannot leverage some partition elimination. If you want to get better performance by skipping the files that don't satisfy some criterion (like specific year or month in this case), use [views on external data](create-use-views.md#partitioned-views).
133134
134135
## External table on appendable files
135136
@@ -153,7 +154,7 @@ WITH (
153154
);
154155
```
155156
156-
The `ALLOW_INCONSISTENT_READS` read option will disable file modification time check during the query lifecycle and read whatever is available in the files that are referenced by the external table. In appendable files, the existing content isn't updated, and only new rows are added. Therefore, the probability of wrong results is minimized compared to the updateable files. This option might enable you to read the frequently appended files without handling the errors.
157+
The `ALLOW_INCONSISTENT_READS` read option disables file modification time check during the query lifecycle and read whatever is available in the files that are referenced by the external table. In appendable files, the existing content isn't updated, and only new rows are added. Therefore, the probability of wrong results is minimized compared to the updateable files. This option might enable you to read the frequently appended files without handling the errors.
157158
158159
This option is available only in the external tables created on CSV file format.
159160
@@ -188,7 +189,7 @@ External tables can't be created on a partitioned folder. Review the other known
188189
External tables in serverless SQL pools don't support partitioning on Delta Lake format. Use [Delta partitioned views](create-use-views.md#delta-lake-partitioned-views) instead of tables if you have partitioned Delta Lake data sets.
189190
190191
> [!IMPORTANT]
191-
> Do not create external tables on partitioned Delta Lake folders even if you see that they might work in some cases. Using unsupported features like external tables on partitioned delta folders might cause issues or instability of the serverless pool. Azure support will not be able to resolve any issue if it is using tables on partitioned folders. You would be asked to transition to [Delta partitioned views](create-use-views.md#delta-lake-partitioned-views) and rewrite your code to use only the supported feature before proceeding with issue resolution.
192+
> Don't create external tables on partitioned Delta Lake folders even if you see that they might work in some cases. Using unsupported features like external tables on partitioned delta folders might cause issues or instability of the serverless pool. Azure support won't be able to resolve any issue if it's using tables on partitioned folders. You would be asked to transition to [Delta partitioned views](create-use-views.md#delta-lake-partitioned-views) and rewrite your code to use only the supported feature before proceeding with issue resolution.
192193
193194
## Use an external table
194195
@@ -197,7 +198,7 @@ You can use [external tables](develop-tables-external-tables.md) in your queries
197198
The following query demonstrates this using the *population* external table we created in previous section. It returns country/region names with their population in 2019 in descending order.
198199
199200
> [!NOTE]
200-
> Change the first line in the query, i.e., [mydbname], so you're using the database you created.
201+
> Change the first line in the query, that is, [mydbname], so you're using the database you created.
201202
202203
```sql
203204
USE [mydbname];

articles/synapse-analytics/sql/develop-tables-external-tables.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ The following example creates a Hadoop external data source in dedicated SQL poo
8787
```sql
8888
CREATE DATABASE SCOPED CREDENTIAL [ADLS_credential]
8989
WITH IDENTITY='SHARED ACCESS SIGNATURE',
90-
SECRET = 'sv=2018-03-28&ss=bf&srt=sco&sp=rl&st=2019-10-14T12%3A10%3A25Z&se=2061-12-31T12%3A10%3A00Z&sig=KlSU2ullCscyTS0An0nozEpo4tO5JAgGBvw%2FJX2lguw%3D'
90+
SECRET = 'sv=2022-11-02&ss=b&srt=co&sp=rl&se=2042-11-26T17:40:55Z&st=2024-11-24T09:40:55Z&spr=https&sig=DKZDuSeZhuCWP9IytWLQwu9shcI5pTJ%2Fw5Crw6fD%2BC8%3D'
9191
GO
9292
CREATE EXTERNAL DATA SOURCE AzureDataLakeStore
9393
WITH
@@ -113,7 +113,7 @@ The following example creates an external data source in serverless or dedicated
113113
```sql
114114
CREATE DATABASE SCOPED CREDENTIAL [sqlondemand]
115115
WITH IDENTITY='SHARED ACCESS SIGNATURE',
116-
SECRET = 'sv=2018-03-28&ss=bf&srt=sco&sp=rl&st=2019-10-14T12%3A10%3A25Z&se=2061-12-31T12%3A10%3A00Z&sig=KlSU2ullCscyTS0An0nozEpo4tO5JAgGBvw%2FJX2lguw%3D'
116+
SECRET = 'sv=2022-11-02&ss=b&srt=co&sp=rl&se=2042-11-26T17:40:55Z&st=2024-11-24T09:40:55Z&spr=https&sig=DKZDuSeZhuCWP9IytWLQwu9shcI5pTJ%2Fw5Crw6fD%2BC8%3D'
117117
GO
118118
CREATE EXTERNAL DATA SOURCE SqlOnDemandDemo WITH (
119119
LOCATION = 'https://sqlondemandstorage.blob.core.windows.net',

0 commit comments

Comments
 (0)