You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/synapse-analytics/sql/develop-openrowset.md
+5-7Lines changed: 5 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -40,10 +40,8 @@ This is a quick and easy way to read the content of the files without pre-config
40
40
TYPE = 'PARQUET') AS file
41
41
```
42
42
43
-
This option enables you to configure location of the storage account in the data source and specify the authentication method that should be used to access storage.
44
-
45
-
> [!IMPORTANT]
46
-
>`OPENROWSET` without `DATA_SOURCE` provides quick and easy way to access the storage files but offers limited authentication options. As an example, Azure AD principal can access files only using their [Azure AD identity](develop-storage-files-storage-access-control.md#user-identity) and cannot access publicly available files. If you need more powerful authentication options, use `DATA_SOURCE` option and define credential that you want to use to access storage.
43
+
> [!IMPORTANT]
44
+
>`OPENROWSET` without `DATA_SOURCE` provides quick and easy way to access the storage files but offers limited authentication options. As an example, Azure AD principal can access files only using their [Azure AD identity](develop-storage-files-storage-access-control.md?tabs=user-identity#database-scoped-credential) and cannot access publicly available files. If you need more powerful authentication options, use `DATA_SOURCE` option and define credential that you want to use to access storage.
47
45
48
46
## Security
49
47
@@ -52,10 +50,10 @@ A database user must have `ADMINISTER BULK OPERATIONS` permission to use the `OP
52
50
The storage administrator must also enable a user to access the files by providing valid SAS token or enabling Azure AD principal to access storage files. Learn more about storage access control in [this article](develop-storage-files-storage-access-control.md).
53
51
54
52
`OPENROWSET` use the following rules to determine how to authenticate to storage:
55
-
-In`OPENROWSET` with `DATA_SOURCE`the authentication mechanism depends on caller type.
56
-
- AAD logins can access files only using their own [Azure AD identity](develop-storage-files-storage-access-control.md#user-identity) if Azure storage allows the Azure AD user to access underlying files (for example, if the caller has Storage Reader permission on storage) and if you [enable Azure AD passthrough authentication](develop-storage-files-storage-access-control.md#force-azure-ad-pass-through) on Synapse SQL service.
53
+
-In`OPENROWSET` with `DATA_SOURCE` authentication mechanism depends on caller type.
54
+
- AAD logins can access files only using their own [Azure AD identity](develop-storage-files-storage-access-control.md?tabs=user-identity#supported-storage-authorization-types) if Azure storage allows the Azure AD user to access underlying files (for example, if the caller has Storage Reader permission on storage) and if you [enable Azure AD passthrough authentication](develop-storage-files-storage-access-control.md#force-azure-ad-pass-through) on Synapse SQL service.
57
55
- SQL logins can also use `OPENROWSET` without `DATA_SOURCE` to access publicly available files, files protected using SAS token or Managed Identity of Synapse workspace. You would need to [create server-scoped credential](develop-storage-files-storage-access-control.md#examples) to allow access to storage files.
58
-
-In`OPENROWSET` with `DATA_SOURCE` authentication mechanism is defined in database scoped credential assigned to the referenced data source. This option enables you to access publicly available storage, or access storage using SAS token, Managed Identity of workspace, or [Azure AD identity of caller](develop-storage-files-storage-access-control.md#user-identity) (if caller is Azure AD principal). If `DATA_SOURCE` references Azure storage that is not public, you would need to [create database-scoped credential](develop-storage-files-storage-access-control.md#examples) and reference it in `DATA SOURCE` to allow access to storage files.
56
+
-In`OPENROWSET` with `DATA_SOURCE` authentication mechanism is defined in database scoped credential assigned to the referenced data source. This option enables you to access publicly available storage, or access storage using SAS token, Managed Identity of workspace, or [Azure AD identity of caller](develop-storage-files-storage-access-control.md?tabs=user-identity#supported-storage-authorization-types) (if caller is Azure AD principal). If `DATA_SOURCE` references Azure storage that is not public, you would need to [create database-scoped credential](develop-storage-files-storage-access-control.md#examples) and reference it in `DATA SOURCE` to allow access to storage files.
59
57
60
58
Caller must have `REFERENCES` permission on credential to use it to authenticate to storage.
# Control storage account access for SQL on-demand (preview)
15
15
16
16
A SQL on-demand query reads files directly from Azure Storage. Permissions to access the files on Azure storage are controlled at two levels:
17
-
-**Storage level** - User should have permission to access underlying storage files. Your storage administrator should allow Azure AD principal to read/write files, or generate SAS key that will be used to to access storage.
17
+
-**Storage level** - User should have permission to access underlying storage files. Your storage administrator should allow Azure AD principal to read/write files, or generate SAS key that will be used to access storage.
18
18
-**SQL service level** - User should have `SELECT` permission to read data from [external table](develop-tables-external-tables.md) or `ADMINISTER BULK ADMIN` permission to execute `OPENROWSET` and also permission to use credentials that will be used to access storage.
19
19
20
20
This article describes the types of credentials you can use and how credential lookup is enacted for SQL and Azure AD users.
@@ -23,22 +23,14 @@ This article describes the types of credentials you can use and how credential l
23
23
24
24
A user that has logged into a SQL on-demand resource must be authorized to access and query the files in Azure Storage. Three authorization types are supported:
> [Azure AD pass-through](#force-azure-ad-pass-through) is the default behavior when you create a workspace. If you use it, you don't need to create credentials for each storage account accessed using Azure AD logins. You can [disable this behavior](#disable-forcing-azure-ad-pass-through).
32
32
33
-
In the table below you can find the available authorization types:
**Shared access signature (SAS)** provides delegated access to resources in a storage account. With SAS, a customer can grant clients access to resources in a storage account without sharing account keys. SAS gives you granular control
44
36
over the type of access you grant to clients who have an SAS, including validity interval, granted permissions, acceptable IP address range, and the acceptable protocol (https/http).
@@ -50,7 +42,7 @@ You can get an SAS token by navigating to the **Azure portal -> Storage Account
50
42
>
51
43
> SAS token: ?sv=2018-03-28&ss=bfqt&srt=sco&sp=rwdlacup&se=2019-04-18T20:42:12Z&st=2019-04-18T12:42:12Z&spr=https&sig=lQHczNvrk1KoYLCpFdSsMANd0ef9BrIPBNJ3VYEIq78%3D
52
44
53
-
### User Identity
45
+
### [User Identity](#tab/user-identity)
54
46
55
47
**User Identity**, also known as "pass-through", is an authorization type where the identity of the Azure AD user that logged into
56
48
SQL on-demand is used to authorize data access. Before accessing the data, the Azure Storage administrator must grant permissions to the Azure AD user. As indicated in the table above, it's not supported for the SQL user type.
@@ -94,12 +86,38 @@ DROP CREDENTIAL [UserIdentity];
94
86
95
87
If you want to re-enable it again, refer to the [force Azure AD pass-through](#force-azure-ad-pass-through) section.
96
88
97
-
### Managed Identity
89
+
### [Managed Identity](#tab/managed-identity)
98
90
99
91
**Managed Identity** is also known as MSI. It's a feature of Azure Active Directory (Azure AD) that provides Azure services for SQL on-demand. Also, it deploys an automatically managed identity in Azure AD. This identity can be used to authorize the request for data access in Azure Storage.
100
92
101
93
Before accessing the data, the Azure Storage administrator must grant permissions to Managed Identity for accessing the data. Granting permissions to Managed Identity is done the same way as granting permission to any other Azure AD user.
102
94
95
+
### [Anonymous access](#tab/public-access)
96
+
97
+
You can access publicly available files placed on Azure storage accounts that allow anonymous access.
98
+
99
+
---
100
+
101
+
### Supported authorization types for databases users
102
+
103
+
In the table below you can find the available authorization types:
To query a file located in Azure Storage, your SQL on-demand end point needs a credential that contains the authentication information. Two types of credentials are used:
@@ -121,7 +139,6 @@ Server-level CREDENTIAL name must match the full path to the storage account (an
121
139
| Azure Data Lake Storage Gen1 | https | <storage_account>.azuredatalakestore.net/webhdfs/v1 |
122
140
| Azure Data Lake Storage Gen2 | https | <storage_account>.dfs.core.windows.net |
123
141
124
-
125
142
> [!NOTE]
126
143
> There is special server-level CREDENTIAL `UserIdentity` that [forces Azure AD pass-through](#force-azure-ad-pass-through).
127
144
@@ -131,17 +148,6 @@ Optionally, to allow a user to create or drop a credential, admin can GRANT/DENY
131
148
GRANT ALTER ANY CREDENTIAL TO [user_name];
132
149
```
133
150
134
-
### Supported storages and authorization types
135
-
136
-
You can use the following combinations of authorization and Azure Storage types:
To use the credential, a user must have `REFERENCES` permission on a specific credential. To grant a `REFERENCES` permission ON a storage_credential for a specific_user, execute:
@@ -156,13 +162,11 @@ To ensure a smooth Azure AD pass-through experience, all users will, by default,
156
162
GRANTREFERENCESON CREDENTIAL::[UserIdentity] TO [public];
157
163
```
158
164
159
-
### Examples
165
+
##Server-scoped credential
160
166
161
-
Depending on the [authorization type](#supported-storage-authorization-types), you can create credentials using the T-SQL syntax below.
162
-
- Server-scoped credentials are used when SQL login calls `OPENROWSET` function without `DATA_SOURCE` to read files on some storage account. The name of server-scoped credential **must** match the URL of Azure storage.
163
-
- Database-scoped credentials are used when any principal calls `OPENROWSET` function with `DATA_SOURCE` or selects data from [external table](develop-tables-external-tables.md) that don't access public files. The database scoped credential don't need to match the name of storage account because it will be explicitly used in DATA SOURCE that defines the location of storage.
167
+
Server-scoped credentials are used when SQL login calls `OPENROWSET` function without `DATA_SOURCE` to read files on some storage account. The name of server-scoped credential **must** match the URL of Azure storage.
164
168
165
-
**Server-scoped credential with Shared Access Signature for Blob Storage**
The following script creates a server-level credential that can be used by `OPENROWSET` function to access any file on Azure storage using SAS token. Create this credential to enable SQL principal that executes `OPENROWSET` function to read files protected with SAS key on the Azure storage that matches URL in credential name.
168
172
@@ -175,7 +179,25 @@ WITH IDENTITY='SHARED ACCESS SIGNATURE'
175
179
GO
176
180
```
177
181
178
-
**Server-scoped credential that allows access to public storage**
182
+
### [User Identity](#tab/user-identity)
183
+
184
+
The following script creates a server-level credential that enables user to impersonate using his Azure AD identity.
185
+
186
+
```sql
187
+
CREATE CREDENTIAL [UserIdentity]
188
+
WITH IDENTITY ='User Identity';
189
+
```
190
+
191
+
### [Managed Identity](#tab/managed-identity)
192
+
193
+
The following script creates a server-level credential that can be used by `OPENROWSET` function to access any file on Azure storage using workspace managed identity.
The following script creates a server-level credential that can be used by `OPENROWSET` function to access any file on publicly available Azure storage. Create this credential to enable SQL principal that executes `OPENROWSET` function to read publicly available files on Azure storage that matches URL in credential name.
181
203
@@ -187,8 +209,14 @@ WITH IDENTITY='SHARED ACCESS SIGNATURE'
187
209
, SECRET ='';
188
210
GO
189
211
```
212
+
---
213
+
214
+
## Database-scoped credential
215
+
216
+
Database-scoped credentials are used when any principal calls `OPENROWSET` function with `DATA_SOURCE` or selects data from [external table](develop-tables-external-tables.md) that don't access public files. The database scoped credential doesn't need to match the name of storage account because it will be explicitly used in DATA SOURCE that defines the location of storage.
**Database-scoped credential with Azure AD Identity**
229
+
### [Azure AD Identity](#tab/user-identity)
202
230
203
231
The following script creates a database-scoped credential that is used by [external table](develop-tables-external-tables.md) and `OPENROWSET` functions that use data source with credential to access storage files using their own Azure AD identity.
204
232
@@ -208,7 +236,7 @@ WITH IDENTITY = 'User Identity';
208
236
GO
209
237
```
210
238
211
-
**Database-scoped credential with Managed Identity**
239
+
### [Managed Identity](#tab/managed-identity)
212
240
213
241
The following script creates a database-scoped credential that can be used to impersonate current Azure AD user as Managed Identity of service.
214
242
@@ -218,7 +246,65 @@ WITH IDENTITY = 'Managed Identity';
218
246
GO
219
247
```
220
248
221
-
The database scoped credential don't need to match the name of storage account because it will be explicitly used in DATA SOURCE that defines the location of storage.
249
+
The database scoped credential doesn't need to match the name of storage account because it will be explicitly used in DATA SOURCE that defines the location of storage.
250
+
251
+
### [Public access](#tab/public-access)
252
+
253
+
Database scoped credential is not required to allow access to publicly available files. Create [data source without database scoped credential](develop-tables-external-tables.md?tabs=sql-ondemand#example-for-create-external-data-source) to access publicly available files on Azure storage.
254
+
255
+
---
256
+
257
+
## Examples
258
+
259
+
**External table that access publicly available data source**
260
+
261
+
Use the following script to create a table that access publicly available data source.
262
+
263
+
```sql
264
+
CREATE EXTERNAL FILE FORMAT [SynapseParquetFormat] WITH ( FORMAT_TYPE = PARQUET)
265
+
GO
266
+
CREATE EXTERNAL DATA SOURCE publicData
267
+
WITH ( LOCATION ='https://****.blob.core.windows.net/public-access' )
**External table that access data source using credential**
275
+
276
+
Modify the following script to create an external table that access Azure storage using SAS token, Azure AD identity of user, or managed identity of workspace.
277
+
278
+
```sql
279
+
-- Create master key in databases with some password (one-off per database)
280
+
CREATE MASTER KEY ENCRYPTION BY PASSWORD ='Y*********0'
281
+
GO
282
+
283
+
-- Create databases scoped credential that use User Identity, Managed Identity, or SAS. User needs to create only database-scoped credentials that should be used to access data source:
284
+
285
+
CREATEDATABASESCOPED CREDENTIAL MyIdentity WITH IDENTITY ='User Identity'
286
+
GO
287
+
CREATEDATABASESCOPED CREDENTIAL WorkspaceIdentity WITH IDENTITY ='Managed Identity'
288
+
GO
289
+
CREATEDATABASESCOPED CREDENTIAL SasCredential WITH IDENTITY ='SHARED ACCESS SIGNATURE', SECRET ='sv=2019-10-1********ZVsTOL0ltEGhf54N8KhDCRfLRI%3D'
290
+
291
+
-- Create data source that one of the credentials above, external file format, and external tables that reference this data source and file format:
292
+
293
+
CREATE EXTERNAL FILE FORMAT [SynapseParquetFormat] WITH ( FORMAT_TYPE = PARQUET)
294
+
GO
295
+
296
+
CREATE EXTERNAL DATA SOURCE mysample
297
+
WITH ( LOCATION ='https://*******.blob.core.windows.net/samples',
298
+
-- Uncomment one of these options depending on authentication method that you want to use to access data source:
0 commit comments