Skip to content

Commit e239166

Browse files
roygararoygara
committed
Merge branch 'master' of https://github.com/MicrosoftDocs/azure-docs-pr into upQuick
2 parents ab3afe6 + 3a1c3ac commit e239166

File tree

1 file changed

+172
-55
lines changed

1 file changed

+172
-55
lines changed

‎articles/synapse-analytics/sql/develop-storage-files-storage-access-control.md

Lines changed: 172 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -14,31 +14,23 @@ ms.reviewer: jrasnick, carlrab
1414
# Control storage account access for SQL on-demand (preview)
1515

1616
A SQL on-demand query reads files directly from Azure Storage. Permissions to access the files on Azure storage are controlled at two levels:
17-
- **Storage level** - User should have permission to access underlying storage files. Your storage administrator should allow Azure AD principal to read/write files, or generate SAS key that will be used to to access storage.
18-
- **SQL service level** - User should have `ADMINISTER BULK ADMIN` permission to execute `OPENROWSET` and also permission to use credentials that will be used to access storage.
17+
- **Storage level** - User should have permission to access underlying storage files. Your storage administrator should allow Azure AD principal to read/write files, or generate SAS key that will be used to access storage.
18+
- **SQL service level** - User should have `SELECT` permission to read data from [external table](develop-tables-external-tables.md) or `ADMINISTER BULK ADMIN` permission to execute `OPENROWSET` and also permission to use credentials that will be used to access storage.
1919

2020
This article describes the types of credentials you can use and how credential lookup is enacted for SQL and Azure AD users.
2121

2222
## Supported storage authorization types
2323

24-
A user that has logged into a SQL on-demand resource must be authorized to access and query the files in Azure Storage. Three authorization types are supported:
24+
A user that has logged into a SQL on-demand resource must be authorized to access and query the files in Azure Storage if the files are not publicly available. Three authorization types are supported:
2525

26-
- [Shared access signature](#shared-access-signature)
27-
- [Managed Identity](#managed-identity)
28-
- [User Identity](#user-identity)
26+
- [User Identity](?tabs=user-identity)
27+
- [Shared access signature](?tabs=shared-access-signature)
28+
- [Managed Identity](?tabs=managed-identity)
2929

3030
> [!NOTE]
3131
> [Azure AD pass-through](#force-azure-ad-pass-through) is the default behavior when you create a workspace. If you use it, you don't need to create credentials for each storage account accessed using Azure AD logins. You can [disable this behavior](#disable-forcing-azure-ad-pass-through).
3232
33-
In the table below you can find the available authorization types:
34-
35-
| Authorization type | *SQL user* | *Azure AD user* |
36-
| ------------------------------------- | ------------- | ----------- |
37-
| [SAS](#shared-access-signature) | Supported | Supported |
38-
| [Managed Identity](#managed-identity) | Not supported | Supported |
39-
| [User Identity](#user-identity) | Not supported | Supported |
40-
41-
### Shared access signature
33+
### [Shared access signature](#tab/shared-access-signature)
4234

4335
**Shared access signature (SAS)** provides delegated access to resources in a storage account. With SAS, a customer can grant clients access to resources in a storage account without sharing account keys. SAS gives you granular control
4436
over the type of access you grant to clients who have an SAS, including validity interval, granted permissions, acceptable IP address range, and the acceptable protocol (https/http).
@@ -50,6 +42,8 @@ You can get an SAS token by navigating to the **Azure portal -> Storage Account
5042
>
5143
> SAS token: ?sv=2018-03-28&ss=bfqt&srt=sco&sp=rwdlacup&se=2019-04-18T20:42:12Z&st=2019-04-18T12:42:12Z&spr=https&sig=lQHczNvrk1KoYLCpFdSsMANd0ef9BrIPBNJ3VYEIq78%3D
5244
45+
You need to create database-scoped or server-scoped credential to enable access using SAS token.
46+
5347
### User Identity
5448

5549
**User Identity**, also known as "pass-through", is an authorization type where the identity of the Azure AD user that logged into
@@ -100,36 +94,21 @@ If you want to re-enable it again, refer to the [force Azure AD pass-through](#f
10094

10195
Before accessing the data, the Azure Storage administrator must grant permissions to Managed Identity for accessing the data. Granting permissions to Managed Identity is done the same way as granting permission to any other Azure AD user.
10296

103-
## Credentials
104-
105-
To query a file located in Azure Storage, your SQL on-demand end point needs a credential that contains the authentication information. Two types of credentials are used:
106-
- Server-level CREDENTIAL is used for ad-hoc queries executed using `OPENROWSET` function. Credential name must match the storage URL.
107-
- DATABASE SCOPED CREDENTIAL is used for external tables. External table references `DATA SOURCE` with the credential that should be used to access storage.
108-
109-
A credential is added by running [CREATE CREDENTIAL](/sql/t-sql/statements/create-credential-transact-sql?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json&view=azure-sqldw-latest). You'll need to provide a CREDENTIAL NAME argument. It must match either part of the path or the whole path to data in Storage (see below).
110-
111-
> [!NOTE]
112-
> The FOR CRYPTOGRAPHIC PROVIDER argument is not supported.
97+
### Anonymous access
11398

114-
For all supported authorization types, credentials can point to an account or a container.
99+
You can access publicly available files placed on Azure storage accounts that [allow anonymous access](/azure/storage/blobs/storage-manage-access-to-resources.md).
115100

116-
Server-level CREDENTIAL name must match the full path to the storage account (and optionally container) in the following format: `<prefix>://<storage_account_path>/<storage_path>`
117-
118-
| External Data Source | Prefix | Storage account path |
119-
| -------------------------- | ------ | --------------------------------------------------- |
120-
| Azure Blob Storage | https | <storage_account>.blob.core.windows.net |
121-
| Azure Data Lake Storage Gen1 | https | <storage_account>.azuredatalakestore.net/webhdfs/v1 |
122-
| Azure Data Lake Storage Gen2 | https | <storage_account>.dfs.core.windows.net |
101+
---
123102

103+
### Supported authorization types for databases users
124104

125-
> [!NOTE]
126-
> There is special server-level CREDENTIAL `UserIdentity` that [forces Azure AD pass-through](#force-azure-ad-pass-through).
127-
128-
Optionally, to allow a user to create or drop a credential, admin can GRANT/DENY ALTER ANY CREDENTIAL permission to a user:
105+
In the table below you can find the available authorization types:
129106

130-
```sql
131-
GRANT ALTER ANY CREDENTIAL TO [user_name];
132-
```
107+
| Authorization type | *SQL user* | *Azure AD user* |
108+
| ------------------------------------- | ------------- | ----------- |
109+
| [User Identity](?tabs=user-identity#supported-storage-authorization-types) | Not supported | Supported |
110+
| [SAS](?tabs=shared-access-signature#supported-storage-authorization-types) | Supported | Supported |
111+
| [Managed Identity](?tabs=managed-identity#supported-storage-authorization-types) | Not supported | Supported |
133112

134113
### Supported storages and authorization types
135114

@@ -141,10 +120,23 @@ You can use the following combinations of authorization and Azure Storage types:
141120
| *Managed Identity* | Supported | Supported | Supported |
142121
| *User Identity* | Supported | Supported | Supported |
143122

123+
## Credentials
124+
125+
To query a file located in Azure Storage, your SQL on-demand end point needs a credential that contains the authentication information. Two types of credentials are used:
126+
- Server-level CREDENTIAL is used for ad-hoc queries executed using `OPENROWSET` function. Credential name must match the storage URL.
127+
- DATABASE SCOPED CREDENTIAL is used for external tables. External table references `DATA SOURCE` with the credential that should be used to access storage.
128+
129+
To allow a user to create or drop a credential, admin can GRANT/DENY ALTER ANY CREDENTIAL permission to a user:
130+
131+
```sql
132+
GRANT ALTER ANY CREDENTIAL TO [user_name];
133+
```
134+
135+
Database users who access external storage must have permission to use credentials.
144136

145137
### Grant permissions to use credential
146138

147-
To use the credential, a user must have REFERENCES permission on a specific credential. To grant a REFERENCES permission ON a storage_credential for a specific_user, execute:
139+
To use the credential, a user must have `REFERENCES` permission on a specific credential. To grant a `REFERENCES` permission ON a storage_credential for a specific_user, execute:
148140

149141
```sql
150142
GRANT REFERENCES ON CREDENTIAL::[storage_credential] TO [specific_user];
@@ -156,15 +148,30 @@ To ensure a smooth Azure AD pass-through experience, all users will, by default,
156148
GRANT REFERENCES ON CREDENTIAL::[UserIdentity] TO [public];
157149
```
158150

159-
### Examples
151+
## Server-scoped credential
152+
153+
Server-scoped credentials are used when SQL login calls `OPENROWSET` function without `DATA_SOURCE` to read files on some storage account. The name of server-scoped credential **must** match the URL of Azure storage. A credential is added by running [CREATE CREDENTIAL](/sql/t-sql/statements/create-credential-transact-sql?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json&view=azure-sqldw-latest). You'll need to provide a CREDENTIAL NAME argument. It must match either part of the path or the whole path to data in Storage (see below).
154+
155+
> [!NOTE]
156+
> The FOR CRYPTOGRAPHIC PROVIDER argument is not supported.
157+
158+
Server-level CREDENTIAL name must match the full path to the storage account (and optionally container) in the following format: `<prefix>://<storage_account_path>/<storage_path>`. Storage account paths are described in the following table:
160159

161-
Depending on the [authorization type](#supported-storage-authorization-types), you can create credentials using the T-SQL syntax below.
162-
- Server-scoped credentials are used when SQL login calls `OPENROWSET` function without `DATA_SOURCE` to read files on some storage account. The name of server-scoped credential **must** match the URL of Azure storage.
163-
- Database-scoped credentials are used when any principal calls `OPENROWSET` function with `DATA_SOURCE` or selects data from external table that don't access public files. The database scoped credential don't need to match the name of storage account because it will be explicitly used in DATA SOURCE that defines the location of storage.
160+
| External Data Source | Prefix | Storage account path |
161+
| -------------------------- | ------ | --------------------------------------------------- |
162+
| Azure Blob Storage | https | <storage_account>.blob.core.windows.net |
163+
| Azure Data Lake Storage Gen1 | https | <storage_account>.azuredatalakestore.net/webhdfs/v1 |
164+
| Azure Data Lake Storage Gen2 | https | <storage_account>.dfs.core.windows.net |
164165

165-
**Server-scoped credential with Shared Access Signature for Blob Storage**
166+
> [!NOTE]
167+
> There is special server-level CREDENTIAL `UserIdentity` that [forces Azure AD pass-through](?tabs=user-identity#force-azure-ad-pass-through).
166168
167-
The following script creates a server-level credential that can be used to access any file on Azure storage using SAS token. Create this credential to enable SQL principal that executes `OPENROWSET` function to read files protected with SAS key on the Azure storage that matches URL in credential name.
169+
Server-scoped credentials enable access to Azure storage using the following authentication types:
170+
171+
### Shared access signature
172+
173+
The following script creates a server-level credential that can be used by `OPENROWSET` function to access any file on Azure storage using SAS token. Create this credential to enable SQL principal that executes `OPENROWSET` function to read files protected
174+
with SAS key on the Azure storage that matches URL in credential name.
168175

169176
Exchange <*mystorageaccountname*> with your actual storage account name, and <*mystorageaccountcontainername*> with the actual container name:
170177

@@ -175,9 +182,27 @@ WITH IDENTITY='SHARED ACCESS SIGNATURE'
175182
GO
176183
```
177184

178-
**Server-scoped credential that allows access to public storage**
185+
### User Identity
186+
187+
The following script creates a server-level credential that enables user to impersonate using Azure AD identity.
188+
189+
```sql
190+
CREATE CREDENTIAL [UserIdentity]
191+
WITH IDENTITY = 'User Identity';
192+
```
193+
194+
### Managed Identity
195+
196+
The following script creates a server-level credential that can be used by `OPENROWSET` function to access any file on Azure storage using workspace managed identity.
197+
198+
```sql
199+
CREATE CREDENTIAL [https://<mystorageaccountname>.blob.core.windows.net/<mystorageaccountcontainername>]
200+
WITH IDENTITY='Managed Identity'
201+
```
202+
203+
### Public access
179204

180-
The following script creates a server-level credential that can be used to access any file on publicly available Azure storage. Create this credential to enable SQL principal that executes `OPENROWSET` function to read publicly available files on Azure storage that matches URL in credential name.
205+
The following script creates a server-level credential that can be used by `OPENROWSET` function to access any file on publicly available Azure storage. Create this credential to enable SQL principal that executes `OPENROWSET` function to read publicly available files on Azure storage that matches URL in credential name.
181206

182207
You would need to have Exchange <*mystorageaccountname*> with your actual storage account name, and <*mystorageaccountcontainername*> with the actual container name:
183208

@@ -187,8 +212,15 @@ WITH IDENTITY='SHARED ACCESS SIGNATURE'
187212
, SECRET = '';
188213
GO
189214
```
215+
---
216+
217+
## Database-scoped credential
190218

191-
**Database-scoped credential with SAS token**
219+
Database-scoped credentials are used when any principal calls `OPENROWSET` function with `DATA_SOURCE` or selects data from [external table](develop-tables-external-tables.md) that don't access public files. The database scoped credential doesn't need to match the name of storage account because it will be explicitly used in DATA SOURCE that defines the location of storage.
220+
221+
Database-scoped credentials enable access to Azure storage using the following authentication types:
222+
223+
### Shared access signature
192224

193225
The following script creates a credential that is used to access files on storage using SAS token specified in the credential.
194226

@@ -198,27 +230,112 @@ WITH IDENTITY = 'SHARED ACCESS SIGNATURE', SECRET = 'sv=2018-03-28&ss=bfqt&srt=s
198230
GO
199231
```
200232

201-
**Database-scoped credential with Azure AD Identity**
233+
### Azure AD Identity
202234

203-
The following script creates a credential that is used by external tables and `OPENROWSET` functions that use data source with credential to access storage files using their own Azure AD identity.
235+
The following script creates a database-scoped credential that is used by [external table](develop-tables-external-tables.md) and `OPENROWSET` functions that use data source with credential to access storage files using their own Azure AD identity.
204236

205237
```sql
206238
CREATE DATABASE SCOPED CREDENTIAL [AzureAD]
207239
WITH IDENTITY = 'User Identity';
208240
GO
209241
```
210242

211-
**Database-scoped credential with Managed Identity**
243+
### Managed Identity
212244

213-
The following script creates a credential that can be used to impersonate current Azure AD user as Managed Identity of service.
245+
The following script creates a database-scoped credential that can be used to impersonate current Azure AD user as Managed Identity of service.
214246

215247
```sql
216248
CREATE DATABASE SCOPED CREDENTIAL [SynapseIdentity]
217249
WITH IDENTITY = 'Managed Identity';
218250
GO
219251
```
220252

221-
The database scoped credential don't need to match the name of storage account because it will be explicitly used in DATA SOURCE that defines the location of storage.
253+
The database scoped credential doesn't need to match the name of storage account because it will be explicitly used in DATA SOURCE that defines the location of storage.
254+
255+
### [Public access](#tab/public-access)
256+
257+
Database scoped credential is not required to allow access to publicly available files. Create [data source without database scoped credential](develop-tables-external-tables.md?tabs=sql-ondemand#example-for-create-external-data-source) to access publicly available files on Azure storage.
258+
259+
---
260+
261+
Database scoped credentials are used in external data sources to specify what authentication method will be used to access this storage:
262+
263+
```sql
264+
CREATE EXTERNAL DATA SOURCE mysample
265+
WITH ( LOCATION = 'https://*******.blob.core.windows.net/samples',
266+
CREDENTIAL = <name of database scoped credential>
267+
)
268+
```
269+
270+
## Examples
271+
272+
**Accessing publicly available data source**
273+
274+
Use the following script to create a table that accesses publicly available data source.
275+
276+
```sql
277+
CREATE EXTERNAL FILE FORMAT [SynapseParquetFormat] WITH ( FORMAT_TYPE = PARQUET)
278+
GO
279+
CREATE EXTERNAL DATA SOURCE publicData
280+
WITH ( LOCATION = 'https://****.blob.core.windows.net/public-access' )
281+
GO
282+
283+
CREATE EXTERNAL TABLE dbo.userPublicData ( [id] int, [first_name] varchar(8000), [last_name] varchar(8000) )
284+
WITH ( LOCATION = 'parquet/user-data/*.parquet', DATA_SOURCE = [publicData], FILE_FORMAT = [SynapseParquetFormat] )
285+
```
286+
287+
Database user can read the content of the files from the data source using external table or [OPENROWSET](develop-openrowset.md) function that references the data source:
288+
289+
```sql
290+
SELECT TOP 10 * FROM dbo.userPublicData;
291+
GO
292+
SELECT TOP 10 * FROM OPENROWSET(BULK 'parquet/user-data/*.parquet', DATA_SOURCE = [mysample], FORMAT=PARQUET) as rows;
293+
GO
294+
```
295+
296+
**Accessing data source using credential**
297+
298+
Modify the following script to create an external table that accesses Azure storage using SAS token, Azure AD identity of user, or managed identity of workspace.
299+
300+
```sql
301+
-- Create master key in databases with some password (one-off per database)
302+
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'Y*********0'
303+
GO
304+
305+
-- Create databases scoped credential that use User Identity, Managed Identity, or SAS. User needs to create only database-scoped credentials that should be used to access data source:
306+
307+
CREATE DATABASE SCOPED CREDENTIAL MyIdentity WITH IDENTITY = 'User Identity'
308+
GO
309+
CREATE DATABASE SCOPED CREDENTIAL WorkspaceIdentity WITH IDENTITY = 'Managed Identity'
310+
GO
311+
CREATE DATABASE SCOPED CREDENTIAL SasCredential WITH IDENTITY = 'SHARED ACCESS SIGNATURE', SECRET = 'sv=2019-10-1********ZVsTOL0ltEGhf54N8KhDCRfLRI%3D'
312+
313+
-- Create data source that one of the credentials above, external file format, and external tables that reference this data source and file format:
314+
315+
CREATE EXTERNAL FILE FORMAT [SynapseParquetFormat] WITH ( FORMAT_TYPE = PARQUET)
316+
GO
317+
318+
CREATE EXTERNAL DATA SOURCE mysample
319+
WITH ( LOCATION = 'https://*******.blob.core.windows.net/samples'
320+
-- Uncomment one of these options depending on authentication method that you want to use to access data source:
321+
--,CREDENTIAL = MyIdentity
322+
--,CREDENTIAL = WorkspaceIdentity
323+
--,CREDENTIAL = SasCredential
324+
)
325+
326+
CREATE EXTERNAL TABLE dbo.userData ( [id] int, [first_name] varchar(8000), [last_name] varchar(8000) )
327+
WITH ( LOCATION = 'parquet/user-data/*.parquet', DATA_SOURCE = [mysample], FILE_FORMAT = [SynapseParquetFormat] )
328+
329+
```
330+
331+
Database user can read the content of the files from the data source using [external table](develop-tables-external-tables.md) or [OPENROWSET](develop-openrowset.md) function that references the data source:
332+
333+
```sql
334+
SELECT TOP 10 * FROM dbo.userdata;
335+
GO
336+
SELECT TOP 10 * FROM OPENROWSET(BULK 'parquet/user-data/*.parquet', DATA_SOURCE = [mysample], FORMAT=PARQUET) as rows;
337+
GO
338+
```
222339

223340
## Next steps
224341

0 commit comments

Comments
 (0)