You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/synapse-analytics/quickstart-serverless-sql-pool.md
+35-41Lines changed: 35 additions & 41 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,68 +1,66 @@
1
1
---
2
2
title: 'Quickstart: Use serverless SQL pool'
3
-
description: In this quickstart, you'll see and learn how easy is to query various types of files using serverless SQL pool.
3
+
description: Learn how to use serverless SQL pool to query various types of files in Azure Storage.
4
4
author: azaricstefan
5
5
ms.service: azure-synapse-analytics
6
6
ms.topic: quickstart
7
7
ms.subservice: sql
8
-
ms.date: 04/15/2020
8
+
ms.date: 02/10/2025
9
9
ms.author: stefanazaric
10
10
ms.reviewer: whhender
11
11
ms.custom: mode-other
12
12
---
13
13
14
14
# Quickstart: Use serverless SQL pool
15
15
16
-
Synapse serverless SQL pool is a serverless query service that enables you to run SQL queries on files placed in Azure Storage. In this quickstart, you'll learn how to query various types of files using serverless SQL pool. Supported formats are listed in[OPENROWSET](sql/develop-openrowset.md).
16
+
Synapse serverless SQL pool is a serverless query service that allows you to run SQL queries on files placed in Azure Storage. In this quickstart, you learn how to query various types of files using serverless SQL pool. For a list of supported formats, see[OPENROWSET](sql/develop-openrowset.md).
17
17
18
-
This quickstart shows querying: CSV, Apache Parquet, and JSON files.
18
+
This quickstart shows how to query CSV, Apache Parquet, and JSON files.
19
19
20
20
## Prerequisites
21
21
22
22
Choose a SQL client to issue queries:
23
23
24
24
-[Azure Synapse Studio](./get-started-create-workspace.md) is a web tool that you can use to browse files in storage and create SQL queries.
25
-
-[Azure Data Studio](sql/get-started-azure-data-studio.md) is a client tool that enables you to run SQL queries and notebooks on your On-demand database.
26
-
-[SQL Server Management Studio](sql/get-started-ssms.md) is a client tool that enables you to run SQL queries on your On-demand database.
25
+
-[Azure Data Studio](sql/get-started-azure-data-studio.md) is a client tool that lets you run SQL queries and notebooks on your on-demand database.
26
+
-[SQL Server Management Studio](sql/get-started-ssms.md) is a client tool that lets you run SQL queries on your on-demand database.
|serverless SQL pool service endpoint address | Used as server name |
33
-
|serverless SQL pool service endpoint region | Used to determine what storage will we use in samples |
32
+
|Serverless SQL pool service endpoint address | Used as server name |
33
+
|Serverless SQL pool service endpoint region | Used to determine what storage to use in samples |
34
34
| Username and password for endpoint access | Used to access endpoint |
35
35
| The database used to create views | Database used as starting point in samples |
36
36
37
37
## First-time setup
38
38
39
39
Before using the samples:
40
40
41
-
- Create database for your views (in case you want to use views)
42
-
- Create credentials to be used by serverless SQL pool to access files in storage
41
+
- Create a database for your views (in case you want to use views).
42
+
- Create credentials to be used by serverless SQL pool to access files in storage.
43
43
44
44
### Create database
45
45
46
-
Create your own database for demo purposes. You'll use this database to create your views and for the sample queries in this article.
46
+
Create your own database for demo purposes. You can use this database to create your views and for the sample queries in this article.
47
47
48
48
> [!NOTE]
49
-
> The databases are used only for view metadata, not for actual data.
50
-
>Write down database name you use for use later in the Quickstart.
49
+
> The databases are used only for view metadata, not for actual data. Write down the database name for use later in the quickstart.
51
50
52
-
Use the following query, changing `mydbname` to a name of your choice:
51
+
Use the following command, changing `mydbname` to a name of your choice:
53
52
54
53
```sql
55
54
CREATEDATABASEmydbname
56
55
```
57
56
58
57
### Create data source
59
58
60
-
To run queries using serverless SQL pool, create data source that serverless SQL pool can use to access files in storage.
61
-
Execute the following code snippet to create data source used in samples in this section:
59
+
To run queries using serverless SQL pool, create a data source that serverless SQL pool can use to access files in storage. Execute the following code snippet to create the data source used in samples in this section:
62
60
63
61
```sql
64
62
-- create master key that will protect the credentials:
65
-
CREATE MASTER KEY ENCRYPTION BY PASSWORD =<enter very strongpasswordhere>
63
+
CREATE MASTER KEY ENCRYPTION BY PASSWORD =<enter-strong-password-here>
66
64
67
65
-- create credentials for containers in our demo storage account
68
66
CREATEDATABASESCOPED CREDENTIAL sqlondemand
@@ -77,9 +75,9 @@ CREATE EXTERNAL DATA SOURCE SqlOnDemandDemo WITH (
77
75
78
76
## Query CSV files
79
77
80
-
The following image is a preview of the file to be queried:
78
+
The following image shows a preview of the file to be queried:
81
79
82
-
80
+
:::image type="content" source="sql/media/query-single-csv-file/population.png" alt-text="Screenshot showing the first 10 rows of the CSV file without header, Windows style new line.":::
83
81
84
82
The following query shows how to read a CSV file that doesn't contain a header row, with Windows-style new line, and comma-delimited columns:
85
83
@@ -102,15 +100,14 @@ WHERE
102
100
country_name ='Luxembourg'AND year =2017
103
101
```
104
102
105
-
You can specify schema at query compilation time.
106
-
For more examples, see how to [query CSV file](sql/query-single-csv-file.md).
103
+
You can specify schema at query compilation time. For more examples, see how to [Query CSV files](sql/query-single-csv-file.md).
107
104
108
105
## Query Parquet files
109
106
110
107
The following sample shows the automatic schema inference capabilities for querying Parquet files. It returns the number of rows in September of 2017 without specifying schema.
111
108
112
109
> [!NOTE]
113
-
> You do not have to specify columns in `OPENROWSET WITH` clause when reading Parquet files. In that case, serverless SQL pool utilizes metadata in the Parquet file and binds columns by name.
110
+
> You don't have to specify columns in `OPENROWSET WITH` clause when reading Parquet files. In that case, serverless SQL pool utilizes metadata in the Parquet file and binds columns by name.
114
111
115
112
```sql
116
113
SELECT COUNT_BIG(*)
@@ -122,13 +119,13 @@ FROM OPENROWSET
122
119
) AS nyc
123
120
```
124
121
125
-
Find more information about [querying parquet files](sql/query-parquet-files.md).
122
+
Find more information, see [Query Parquet files using serverless SQL pool](sql/query-parquet-files.md).
126
123
127
124
## Query JSON files
128
125
129
126
### JSON sample file
130
127
131
-
Files are stored in *json* container, folder *books*, and contain single book entry with following structure:
128
+
Files are stored in a *json* container, using folder *books*, and contain a single book entry with the following structure:
132
129
133
130
```json
134
131
{
@@ -146,9 +143,9 @@ Files are stored in *json* container, folder *books*, and contain single book en
146
143
}
147
144
```
148
145
149
-
### Query JSON files
146
+
### Sample query
150
147
151
-
The following query shows how to use [JSON_VALUE](/sql/t-sql/functions/json-value-transact-sql?view=azure-sqldw-latest&preserve-view=true) to retrieve scalar values (title, publisher) from a book with the title *Probabilistic and Statistical Methods in Cryptology, An Introduction by Selected articles*:
148
+
The following query shows how to use [JSON_VALUE](/sql/t-sql/functions/json-value-transact-sql?view=azure-sqldw-latest&preserve-view=true) to retrieve scalar values (title, publisher) from a book with the title *Probabilistic and Statistical Methods in Cryptology, An Introduction by selected topics*:
152
149
153
150
```sql
154
151
SELECT
@@ -167,23 +164,20 @@ FROM OPENROWSET
167
164
WITH
168
165
( jsonContent varchar(8000) ) AS [r]
169
166
WHERE
170
-
JSON_VALUE(jsonContent, '$.title') ='Probabilistic and Statistical Methods in Cryptology, An Introduction by Selected Topics'
167
+
JSON_VALUE(jsonContent, '$.title') ='Probabilistic and Statistical Methods in Cryptology, An Introduction by selected topics'
171
168
```
172
169
173
170
> [!IMPORTANT]
174
-
> We are reading the entire JSON file as single row/column. So, FIELDTERMINATOR, FIELDQUOTE, and ROWTERMINATOR are set to 0x0b because we do not expect to find it in the file.
171
+
> We read the entire JSON file as a single row or column. So FIELDTERMINATOR, FIELDQUOTE, and ROWTERMINATOR are set to 0x0b because we don't expect to find it in the file.
175
172
176
-
## Next steps
173
+
## Related content
177
174
178
-
You're now ready to continue on with the following articles:
179
-
180
-
-[Query single CSV file](sql/query-single-csv-file.md)
181
-
-[Query folders and multiple CSV files](sql/query-folders-multiple-csv-files.md)
175
+
-[Query CSV files](sql/query-single-csv-file.md)
176
+
-[Query folders and multiple files](sql/query-folders-multiple-csv-files.md)
182
177
-[Query specific files](sql/query-specific-files.md)
title: Data exfiltration protection for Azure Synapse Analytics workspaces
3
-
description: This article will explain data exfiltration protection in Azure Synapse Analytics
2
+
title: Data exfiltration protection for Azure Synapse Analytics
3
+
description: Learn how to use data exfiltration protection in Azure Synapse Analytics workspaces.
4
4
author: WilliamDAssafMSFT
5
5
ms.service: azure-synapse-analytics
6
-
ms.topic: conceptual
6
+
ms.topic: concept-article
7
7
ms.subservice: security
8
-
ms.date: 10/17/2022
8
+
ms.date: 02/10/2025
9
9
ms.author: wiassaf
10
10
ms.reviewer: whhender
11
11
---
12
+
12
13
# Data exfiltration protection for Azure Synapse Analytics workspaces
13
-
This article will explain data exfiltration protection in Azure Synapse Analytics
14
14
15
-
## Securing data egress from Synapse workspaces
16
-
Azure Synapse Analytics workspaces support enabling data exfiltration protection for workspaces. With exfiltration protection, you can guard against malicious insiders accessing your Azure resources and exfiltrating sensitive data to locations outside of your organization’s scope.
17
-
At the time of workspace creation, you can choose to configure the workspace with a managed virtual network and additional protection against data exfiltration. When a workspace is created with a [managed virtual network](./synapse-workspace-managed-vnet.md), Data integration and Spark resources are deployed in the managed virtual network. The workspace’s dedicated SQL pools and serverless SQL pools have multi-tenant capabilities and as such, need to exist outside the managed virtual network. For workspaces with data exfiltration protection, resources within the managed virtual network always communicate over [managed private endpoints](./synapse-workspace-managed-private-endpoints.md). When data exfiltration protection is enabled, Synapse SQL resources can connect to and query any authorized Azure Storage using OPENROWSETS or EXTERNAL TABLE, since the ingress traffic is not controlled by the data exfiltration protection. However, the egress traffic, for example [CREATE EXTERNAL TABLE AS SELECT](/sql/t-sql/statements/create-external-table-as-select-transact-sql?view=azure-sqldw-latest&preserve-view=true) or using ERRORFILE argument in [COPY INTO](/sql/t-sql/statements/copy-into-transact-sql?view=azure-sqldw-latest&preserve-view=true) command to output data to the external storage account will be controlled by the data exfiltration protection. Therefore, it also requires to create the managed private endpoint for the target storage account to unblock the egress traffic to it.
15
+
Azure Synapse Analytics workspaces support data exfiltration protection for workspaces. With exfiltration protection, you can guard against malicious insiders accessing your Azure resources and exfiltrating sensitive data to locations outside of your organization's scope.
18
16
19
-
> [!Note]
20
-
> You cannot change the workspace configuration for managed virtual network and data exfiltration protection after the workspace is created.
17
+
## Secure data egress from Synapse workspaces
21
18
22
-
## Managing Synapse workspace data egress to approved targets
23
-
After the workspace is created with data exfiltration protection enabled, the owners of the workspace resource can manage the list of approved Microsoft Entra tenants for the workspace. Users with the [right permissions](./synapse-workspace-access-control-overview.md) on the workspace can use the Synapse Studio to create managed private endpoint connection requests to resources in the workspace’s approved Microsoft Entra tenants. Managed private endpoint creation will be blocked if the user attempts to create a private endpoint connection to a resource in an unapproved tenant.
19
+
At the time of workspace creation, you can choose to configure the workspace with a managed virtual network and additional protection against data exfiltration. When a workspace is created with a [managed virtual network](./synapse-workspace-managed-vnet.md), data integration and Spark resources are deployed in the managed virtual network. The workspace's dedicated SQL pools and serverless SQL pools have multitenant capabilities and as such, need to exist outside the managed virtual network.
24
20
25
-
## Sample workspace with data exfiltration protection enabled
26
-
Let us use an example to illustrate data exfiltration protection for Synapse workspaces. Contoso has Azure resources in Tenant A and Tenant B and there is a need for these resources to connect securely. A Synapse workspace has been created in Tenant A with Tenant B added as an approved Microsoft Entra tenant. The diagram shows private endpoint connections to Azure Storage accounts in Tenant A and Tenant B that have been approved by the Storage account owners. The diagram also shows blocked private endpoint creation. The creation of this private endpoint was blocked as it targeted an Azure Storage account in the Fabrikam Microsoft Entra tenant, which is not an approved Microsoft Entra tenant for Contoso’s workspace.
21
+
For workspaces with data exfiltration protection, resources within the managed virtual network always communicate over [managed private endpoints](./synapse-workspace-managed-private-endpoints.md). When data exfiltration protection is enabled, Synapse SQL resources can connect to and query any authorized Azure Storage using OPENROWSETS or EXTERNAL TABLE. Data exfiltration protection doesn't control ingress traffic.
27
22
28
-
:::image type="content" source="media/workspace-data-exfiltration-protection/workspace-data-exfiltration-protection-diagram.png" alt-text="This diagram shows how data exfiltration protection is implemented for Synapse workspaces" lightbox="./media/workspace-data-exfiltration-protection/workspace-data-exfiltration-protection-diagram.png":::
23
+
However, data exfiltration protection does control egress traffic. For example, [CREATE EXTERNAL TABLE AS SELECT](/sql/t-sql/statements/create-external-table-as-select-transact-sql?view=azure-sqldw-latest&preserve-view=true) or using ERRORFILE argument in [COPY INTO](/sql/t-sql/statements/copy-into-transact-sql?view=azure-sqldw-latest&preserve-view=true) command to output data to the external storage account are blocked. Therefore, you need to create a managed private endpoint for the target storage account to unblock the egress traffic to it.
29
24
30
-
>[!IMPORTANT]
31
-
>
32
-
> Resources in tenants other than the workspace's tenant must not have blocking firewall rules in place for the SQL pools to connect to them. Resources within the workspace’s managed virtual network, such as Spark clusters, can connect over managed private links to firewall-protected resources.
33
-
> >
25
+
> [!NOTE]
26
+
> You can't change the workspace configuration for managed virtual network and data exfiltration protection after the workspace is created.
34
27
35
-
## Next Steps
28
+
## Manage Synapse workspace data egress to approved targets
36
29
37
-
Learn how to [create a workspace with data exfiltration protection enabled](./how-to-create-a-workspace-with-data-exfiltration-protection.md)
30
+
After the workspace is created with data exfiltration protection enabled, the owners of the workspace resource can manage the list of approved Microsoft Entra tenants for the workspace. Users with the [right permissions](./synapse-workspace-access-control-overview.md) on the workspace can use the Synapse Studio to create managed private endpoint connection requests to resources in the workspace’s approved Microsoft Entra tenants. Managed private endpoint creation is blocked if the user attempts to create a private endpoint connection to a resource in an unapproved tenant.
31
+
32
+
## Sample workspace with data exfiltration protection enabled
38
33
39
-
Learn more about [Managed workspace Virtual Network](./synapse-workspace-managed-vnet.md)
34
+
Consider the following example that illustrates data exfiltration protection for Synapse workspaces. A company called Contoso has Azure resources in Tenant A and Tenant B, and there's a need for these resources to connect securely. A Synapse workspace has been created in Tenant A with Tenant B added as an approved Microsoft Entra tenant.
35
+
36
+
The following diagram shows private endpoint connections to Azure Storage accounts in Tenant A and Tenant B that are approved by the storage account owners. The diagram also shows blocked private endpoint creation. The creation of this private endpoint was blocked as it targeted an Azure Storage account in the Fabrikam Microsoft Entra tenant, which isn't an approved Microsoft Entra tenant for Contoso's workspace.
37
+
38
+
:::image type="content" source="media/workspace-data-exfiltration-protection/workspace-data-exfiltration-protection-diagram.png" alt-text="Diagram showing how data exfiltration protection is implemented for Synapse workspaces." lightbox="./media/workspace-data-exfiltration-protection/workspace-data-exfiltration-protection-diagram.png":::
39
+
40
+
>[!IMPORTANT]
41
+
> Resources in tenants other than the workspace's tenant must not have firewall rules that block connection to the SQL pools. Resources within the workspace's managed virtual network, such as Spark clusters, can connect over managed private links to firewall-protected resources.
40
42
41
-
Learn more about [Managed private endpoints](./synapse-workspace-managed-private-endpoints.md)
43
+
## Related content
42
44
43
-
[Create Managed private endpoints to your data sources](./how-to-create-managed-private-endpoints.md)
45
+
-[Create a workspace with data exfiltration protection enabled](./how-to-create-a-workspace-with-data-exfiltration-protection.md)
0 commit comments