Skip to content

Commit f99b52d

Browse files
committed
resolving merge conflict
2 parents 291fdc5 + 9138cb7 commit f99b52d

File tree

2 files changed

+128
-44
lines changed

2 files changed

+128
-44
lines changed

articles/synapse-analytics/sql/develop-openrowset.md

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -40,10 +40,8 @@ This is a quick and easy way to read the content of the files without pre-config
4040
TYPE = 'PARQUET') AS file
4141
```
4242

43-
This option enables you to configure location of the storage account in the data source and specify the authentication method that should be used to access storage.
44-
45-
> [!IMPORTANT]
46-
> `OPENROWSET` without `DATA_SOURCE` provides quick and easy way to access the storage files but offers limited authentication options. As an example, Azure AD principal can access files only using their [Azure AD identity](develop-storage-files-storage-access-control.md#user-identity) and cannot access publicly available files. If you need more powerful authentication options, use `DATA_SOURCE` option and define credential that you want to use to access storage.
43+
> [!IMPORTANT]
44+
> `OPENROWSET` without `DATA_SOURCE` provides quick and easy way to access the storage files but offers limited authentication options. As an example, Azure AD principal can access files only using their [Azure AD identity](develop-storage-files-storage-access-control.md?tabs=user-identity#database-scoped-credential) and cannot access publicly available files. If you need more powerful authentication options, use `DATA_SOURCE` option and define credential that you want to use to access storage.
4745

4846
## Security
4947

@@ -52,10 +50,10 @@ A database user must have `ADMINISTER BULK OPERATIONS` permission to use the `OP
5250
The storage administrator must also enable a user to access the files by providing valid SAS token or enabling Azure AD principal to access storage files. Learn more about storage access control in [this article](develop-storage-files-storage-access-control.md).
5351

5452
`OPENROWSET` use the following rules to determine how to authenticate to storage:
55-
- In `OPENROWSET` with `DATA_SOURCE` the authentication mechanism depends on caller type.
56-
- AAD logins can access files only using their own [Azure AD identity](develop-storage-files-storage-access-control.md#user-identity) if Azure storage allows the Azure AD user to access underlying files (for example, if the caller has Storage Reader permission on storage) and if you [enable Azure AD passthrough authentication](develop-storage-files-storage-access-control.md#force-azure-ad-pass-through) on Synapse SQL service.
53+
- In `OPENROWSET` with `DATA_SOURCE` authentication mechanism depends on caller type.
54+
- AAD logins can access files only using their own [Azure AD identity](develop-storage-files-storage-access-control.md?tabs=user-identity#supported-storage-authorization-types) if Azure storage allows the Azure AD user to access underlying files (for example, if the caller has Storage Reader permission on storage) and if you [enable Azure AD passthrough authentication](develop-storage-files-storage-access-control.md#force-azure-ad-pass-through) on Synapse SQL service.
5755
- SQL logins can also use `OPENROWSET` without `DATA_SOURCE` to access publicly available files, files protected using SAS token or Managed Identity of Synapse workspace. You would need to [create server-scoped credential](develop-storage-files-storage-access-control.md#examples) to allow access to storage files.
58-
- In `OPENROWSET` with `DATA_SOURCE` authentication mechanism is defined in database scoped credential assigned to the referenced data source. This option enables you to access publicly available storage, or access storage using SAS token, Managed Identity of workspace, or [Azure AD identity of caller](develop-storage-files-storage-access-control.md#user-identity) (if caller is Azure AD principal). If `DATA_SOURCE` references Azure storage that is not public, you would need to [create database-scoped credential](develop-storage-files-storage-access-control.md#examples) and reference it in `DATA SOURCE` to allow access to storage files.
56+
- In `OPENROWSET` with `DATA_SOURCE` authentication mechanism is defined in database scoped credential assigned to the referenced data source. This option enables you to access publicly available storage, or access storage using SAS token, Managed Identity of workspace, or [Azure AD identity of caller](develop-storage-files-storage-access-control.md?tabs=user-identity#supported-storage-authorization-types) (if caller is Azure AD principal). If `DATA_SOURCE` references Azure storage that is not public, you would need to [create database-scoped credential](develop-storage-files-storage-access-control.md#examples) and reference it in `DATA SOURCE` to allow access to storage files.
5957

6058
Caller must have `REFERENCES` permission on credential to use it to authenticate to storage.
6159

articles/synapse-analytics/sql/develop-storage-files-storage-access-control.md

Lines changed: 123 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ ms.reviewer: jrasnick, carlrab
1414
# Control storage account access for SQL on-demand (preview)
1515

1616
A SQL on-demand query reads files directly from Azure Storage. Permissions to access the files on Azure storage are controlled at two levels:
17-
- **Storage level** - User should have permission to access underlying storage files. Your storage administrator should allow Azure AD principal to read/write files, or generate SAS key that will be used to to access storage.
17+
- **Storage level** - User should have permission to access underlying storage files. Your storage administrator should allow Azure AD principal to read/write files, or generate SAS key that will be used to access storage.
1818
- **SQL service level** - User should have `SELECT` permission to read data from [external table](develop-tables-external-tables.md) or `ADMINISTER BULK ADMIN` permission to execute `OPENROWSET` and also permission to use credentials that will be used to access storage.
1919

2020
This article describes the types of credentials you can use and how credential lookup is enacted for SQL and Azure AD users.
@@ -23,22 +23,14 @@ This article describes the types of credentials you can use and how credential l
2323

2424
A user that has logged into a SQL on-demand resource must be authorized to access and query the files in Azure Storage. Three authorization types are supported:
2525

26-
- [User Identity](#user-identity)
27-
- [Shared access signature](#shared-access-signature)
28-
- [Managed Identity](#managed-identity)
26+
- [User Identity](?tabs=user-identity)
27+
- [Shared access signature](?tabs=shared-access-signature)
28+
- [Managed Identity](?tabs=managed-identity)
2929

3030
> [!NOTE]
3131
> [Azure AD pass-through](#force-azure-ad-pass-through) is the default behavior when you create a workspace. If you use it, you don't need to create credentials for each storage account accessed using Azure AD logins. You can [disable this behavior](#disable-forcing-azure-ad-pass-through).
3232
33-
In the table below you can find the available authorization types:
34-
35-
| Authorization type | *SQL user* | *Azure AD user* |
36-
| ------------------------------------- | ------------- | ----------- |
37-
| [User Identity](#user-identity) | Not supported | Supported |
38-
| [SAS](#shared-access-signature) | Supported | Supported |
39-
| [Managed Identity](#managed-identity) | Not supported | Supported |
40-
41-
### Shared access signature
33+
### [Shared access signature](#tab/shared-access-signature)
4234

4335
**Shared access signature (SAS)** provides delegated access to resources in a storage account. With SAS, a customer can grant clients access to resources in a storage account without sharing account keys. SAS gives you granular control
4436
over the type of access you grant to clients who have an SAS, including validity interval, granted permissions, acceptable IP address range, and the acceptable protocol (https/http).
@@ -50,7 +42,7 @@ You can get an SAS token by navigating to the **Azure portal -> Storage Account
5042
>
5143
> SAS token: ?sv=2018-03-28&ss=bfqt&srt=sco&sp=rwdlacup&se=2019-04-18T20:42:12Z&st=2019-04-18T12:42:12Z&spr=https&sig=lQHczNvrk1KoYLCpFdSsMANd0ef9BrIPBNJ3VYEIq78%3D
5244
53-
### User Identity
45+
### [User Identity](#tab/user-identity)
5446

5547
**User Identity**, also known as "pass-through", is an authorization type where the identity of the Azure AD user that logged into
5648
SQL on-demand is used to authorize data access. Before accessing the data, the Azure Storage administrator must grant permissions to the Azure AD user. As indicated in the table above, it's not supported for the SQL user type.
@@ -94,12 +86,38 @@ DROP CREDENTIAL [UserIdentity];
9486

9587
If you want to re-enable it again, refer to the [force Azure AD pass-through](#force-azure-ad-pass-through) section.
9688

97-
### Managed Identity
89+
### [Managed Identity](#tab/managed-identity)
9890

9991
**Managed Identity** is also known as MSI. It's a feature of Azure Active Directory (Azure AD) that provides Azure services for SQL on-demand. Also, it deploys an automatically managed identity in Azure AD. This identity can be used to authorize the request for data access in Azure Storage.
10092

10193
Before accessing the data, the Azure Storage administrator must grant permissions to Managed Identity for accessing the data. Granting permissions to Managed Identity is done the same way as granting permission to any other Azure AD user.
10294

95+
### [Anonymous access](#tab/public-access)
96+
97+
You can access publicly available files placed on Azure storage accounts that allow anonymous access.
98+
99+
---
100+
101+
### Supported authorization types for databases users
102+
103+
In the table below you can find the available authorization types:
104+
105+
| Authorization type | *SQL user* | *Azure AD user* |
106+
| ------------------------------------- | ------------- | ----------- |
107+
| [User Identity](?tabs=user-identity#supported-storage-authorization-types) | Not supported | Supported |
108+
| [SAS](?tabs=shared-access-signature#supported-storage-authorization-types) | Supported | Supported |
109+
| [Managed Identity](?tabs=managed-identity#supported-storage-authorization-types) | Not supported | Supported |
110+
111+
### Supported storages and authorization types
112+
113+
You can use the following combinations of authorization and Azure Storage types:
114+
115+
| | Blob Storage | ADLS Gen1 | ADLS Gen2 |
116+
| ------------------- | ------------ | -------------- | ----------- |
117+
| *SAS* | Supported | Not supported | Supported |
118+
| *Managed Identity* | Supported | Supported | Supported |
119+
| *User Identity* | Supported | Supported | Supported |
120+
103121
## Credentials
104122

105123
To query a file located in Azure Storage, your SQL on-demand end point needs a credential that contains the authentication information. Two types of credentials are used:
@@ -121,7 +139,6 @@ Server-level CREDENTIAL name must match the full path to the storage account (an
121139
| Azure Data Lake Storage Gen1 | https | <storage_account>.azuredatalakestore.net/webhdfs/v1 |
122140
| Azure Data Lake Storage Gen2 | https | <storage_account>.dfs.core.windows.net |
123141

124-
125142
> [!NOTE]
126143
> There is special server-level CREDENTIAL `UserIdentity` that [forces Azure AD pass-through](#force-azure-ad-pass-through).
127144
@@ -131,17 +148,6 @@ Optionally, to allow a user to create or drop a credential, admin can GRANT/DENY
131148
GRANT ALTER ANY CREDENTIAL TO [user_name];
132149
```
133150

134-
### Supported storages and authorization types
135-
136-
You can use the following combinations of authorization and Azure Storage types:
137-
138-
| | Blob Storage | ADLS Gen1 | ADLS Gen2 |
139-
| ------------------- | ------------ | -------------- | ----------- |
140-
| *SAS* | Supported | Not supported | Supported |
141-
| *Managed Identity* | Supported | Supported | Supported |
142-
| *User Identity* | Supported | Supported | Supported |
143-
144-
145151
### Grant permissions to use credential
146152

147153
To use the credential, a user must have `REFERENCES` permission on a specific credential. To grant a `REFERENCES` permission ON a storage_credential for a specific_user, execute:
@@ -156,13 +162,11 @@ To ensure a smooth Azure AD pass-through experience, all users will, by default,
156162
GRANT REFERENCES ON CREDENTIAL::[UserIdentity] TO [public];
157163
```
158164

159-
### Examples
165+
## Server-scoped credential
160166

161-
Depending on the [authorization type](#supported-storage-authorization-types), you can create credentials using the T-SQL syntax below.
162-
- Server-scoped credentials are used when SQL login calls `OPENROWSET` function without `DATA_SOURCE` to read files on some storage account. The name of server-scoped credential **must** match the URL of Azure storage.
163-
- Database-scoped credentials are used when any principal calls `OPENROWSET` function with `DATA_SOURCE` or selects data from [external table](develop-tables-external-tables.md) that don't access public files. The database scoped credential don't need to match the name of storage account because it will be explicitly used in DATA SOURCE that defines the location of storage.
167+
Server-scoped credentials are used when SQL login calls `OPENROWSET` function without `DATA_SOURCE` to read files on some storage account. The name of server-scoped credential **must** match the URL of Azure storage.
164168

165-
**Server-scoped credential with Shared Access Signature for Blob Storage**
169+
### [Shared access signature](#tab/shared-access-signature)
166170

167171
The following script creates a server-level credential that can be used by `OPENROWSET` function to access any file on Azure storage using SAS token. Create this credential to enable SQL principal that executes `OPENROWSET` function to read files protected with SAS key on the Azure storage that matches URL in credential name.
168172

@@ -175,7 +179,25 @@ WITH IDENTITY='SHARED ACCESS SIGNATURE'
175179
GO
176180
```
177181

178-
**Server-scoped credential that allows access to public storage**
182+
### [User Identity](#tab/user-identity)
183+
184+
The following script creates a server-level credential that enables user to impersonate using his Azure AD identity.
185+
186+
```sql
187+
CREATE CREDENTIAL [UserIdentity]
188+
WITH IDENTITY = 'User Identity';
189+
```
190+
191+
### [Managed Identity](#tab/managed-identity)
192+
193+
The following script creates a server-level credential that can be used by `OPENROWSET` function to access any file on Azure storage using workspace managed identity.
194+
195+
```sql
196+
CREATE CREDENTIAL [https://<mystorageaccountname>.blob.core.windows.net/<mystorageaccountcontainername>]
197+
WITH IDENTITY='Managed Identity'
198+
```
199+
200+
### [Public access](#tab/public-access)
179201

180202
The following script creates a server-level credential that can be used by `OPENROWSET` function to access any file on publicly available Azure storage. Create this credential to enable SQL principal that executes `OPENROWSET` function to read publicly available files on Azure storage that matches URL in credential name.
181203

@@ -187,8 +209,14 @@ WITH IDENTITY='SHARED ACCESS SIGNATURE'
187209
, SECRET = '';
188210
GO
189211
```
212+
---
213+
214+
## Database-scoped credential
215+
216+
Database-scoped credentials are used when any principal calls `OPENROWSET` function with `DATA_SOURCE` or selects data from [external table](develop-tables-external-tables.md) that don't access public files. The database scoped credential doesn't need to match the name of storage account because it will be explicitly used in DATA SOURCE that defines the location of storage.
190217

191-
**Database-scoped credential with SAS token**
218+
219+
### [Shared access signature](#tab/shared-access-signature)
192220

193221
The following script creates a credential that is used to access files on storage using SAS token specified in the credential.
194222

@@ -198,7 +226,7 @@ WITH IDENTITY = 'SHARED ACCESS SIGNATURE', SECRET = 'sv=2018-03-28&ss=bfqt&srt=s
198226
GO
199227
```
200228

201-
**Database-scoped credential with Azure AD Identity**
229+
### [Azure AD Identity](#tab/user-identity)
202230

203231
The following script creates a database-scoped credential that is used by [external table](develop-tables-external-tables.md) and `OPENROWSET` functions that use data source with credential to access storage files using their own Azure AD identity.
204232

@@ -208,7 +236,7 @@ WITH IDENTITY = 'User Identity';
208236
GO
209237
```
210238

211-
**Database-scoped credential with Managed Identity**
239+
### [Managed Identity](#tab/managed-identity)
212240

213241
The following script creates a database-scoped credential that can be used to impersonate current Azure AD user as Managed Identity of service.
214242

@@ -218,7 +246,65 @@ WITH IDENTITY = 'Managed Identity';
218246
GO
219247
```
220248

221-
The database scoped credential don't need to match the name of storage account because it will be explicitly used in DATA SOURCE that defines the location of storage.
249+
The database scoped credential doesn't need to match the name of storage account because it will be explicitly used in DATA SOURCE that defines the location of storage.
250+
251+
### [Public access](#tab/public-access)
252+
253+
Database scoped credential is not required to allow access to publicly available files. Create [data source without database scoped credential](develop-tables-external-tables.md?tabs=sql-ondemand#example-for-create-external-data-source) to access publicly available files on Azure storage.
254+
255+
---
256+
257+
## Examples
258+
259+
**External table that access publicly available data source**
260+
261+
Use the following script to create a table that access publicly available data source.
262+
263+
```sql
264+
CREATE EXTERNAL FILE FORMAT [SynapseParquetFormat] WITH ( FORMAT_TYPE = PARQUET)
265+
GO
266+
CREATE EXTERNAL DATA SOURCE publicData
267+
WITH ( LOCATION = 'https://****.blob.core.windows.net/public-access' )
268+
GO
269+
270+
CREATE EXTERNAL TABLE dbo.userPublicData ( [id] int, [first_name] varchar(8000), [last_name] varchar(8000) )
271+
WITH ( LOCATION = 'parquet/user-data/userdata.parquet', DATA_SOURCE = [publicData], FILE_FORMAT = [SynapseParquetFormat] )
272+
```
273+
274+
**External table that access data source using credential**
275+
276+
Modify the following script to create an external table that access Azure storage using SAS token, Azure AD identity of user, or managed identity of workspace.
277+
278+
```sql
279+
-- Create master key in databases with some password (one-off per database)
280+
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'Y*********0'
281+
GO
282+
283+
-- Create databases scoped credential that use User Identity, Managed Identity, or SAS. User needs to create only database-scoped credentials that should be used to access data source:
284+
285+
CREATE DATABASE SCOPED CREDENTIAL MyIdentity WITH IDENTITY = 'User Identity'
286+
GO
287+
CREATE DATABASE SCOPED CREDENTIAL WorkspaceIdentity WITH IDENTITY = 'Managed Identity'
288+
GO
289+
CREATE DATABASE SCOPED CREDENTIAL SasCredential WITH IDENTITY = 'SHARED ACCESS SIGNATURE', SECRET = 'sv=2019-10-1********ZVsTOL0ltEGhf54N8KhDCRfLRI%3D'
290+
291+
-- Create data source that one of the credentials above, external file format, and external tables that reference this data source and file format:
292+
293+
CREATE EXTERNAL FILE FORMAT [SynapseParquetFormat] WITH ( FORMAT_TYPE = PARQUET)
294+
GO
295+
296+
CREATE EXTERNAL DATA SOURCE mysample
297+
WITH ( LOCATION = 'https://*******.blob.core.windows.net/samples',
298+
-- Uncomment one of these options depending on authentication method that you want to use to access data source:
299+
--,CREDENTIAL = MyIdentity
300+
--,CREDENTIAL = WorkspaceIdentity
301+
--,CREDENTIAL = SasCredential
302+
)
303+
304+
CREATE EXTERNAL TABLE dbo.userData ( [id] int, [first_name] varchar(8000), [last_name] varchar(8000) )
305+
WITH ( LOCATION = 'parquet/user-data/*.parquet', DATA_SOURCE = [mysample], FILE_FORMAT = [SynapseParquetFormat] )
306+
307+
```
222308

223309
## Next steps
224310

0 commit comments

Comments
 (0)