Skip to content

Commit 8134333

Browse files
Merge pull request #224466 from whhender/private-endpoint-shir-options
Private endpoint shir options
2 parents 462ab79 + 219e542 commit 8134333

File tree

2 files changed

+46
-45
lines changed

2 files changed

+46
-45
lines changed

articles/purview/register-scan-adls-gen2.md

Lines changed: 23 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -73,33 +73,30 @@ It's important to register the data source in Microsoft Purview prior to setting
7373

7474
> [!TIP]
7575
> To troubleshoot any issues with scanning:
76-
> 1. Confirm you have followed all [**prerequisites for scanning**](#prerequisites-for-scan).
76+
> 1. Confirm you have properly set up [**authentication for scanning**](#authentication-for-a-scan)
7777
> 1. Review our [**scan troubleshooting documentation**](troubleshoot-connections.md).
7878
79-
### Prerequisites for scan
79+
### Authentication for a scan
8080

81-
In order to have access to scan the data source, an authentication method in the ADLS Gen2 Storage account needs to be configured.
82-
The following options are supported:
81+
Your Azure network may allow for communications between your Azure resources, but if you've set up firewalls, private endpoints, or virtual networks within Azure, you'll need to follow one of these configurations below.
8382

84-
> [!Note]
85-
> If you have firewall enabled for the storage account, you must use managed identity authentication method when setting up a scan.
83+
|Networking constraints |Integration runtime type |Available credential types |
84+
|---------|---------|---------|
85+
|No private endpoints or firewalls | Azure IR | Managed identity (Recommended), service principal, or account key|
86+
|Firewall enabled but no private endpoints| Azure IR | Managed identity |
87+
|Private endpoints enabled | *Self-Hosted IR | Service principal, account key|
8688

87-
* **System-assigned managed identity (Recommended)** - As soon as the Microsoft Purview Account is created, a system-assigned managed identity (SAMI) is created automatically in Azure AD tenant. Depending on the type of resource, specific RBAC role assignments are required for the Microsoft Purview system-assigned managed identity (SAMI) to perform the scans.
88-
89-
* **User-assigned managed identity** (preview) - Similar to a system managed identity, a user-assigned managed identity (UAMI) is a credential resource that can be used to allow Microsoft Purview to authenticate against Azure Active Directory. For more information, you can see our [User-assigned managed identity guide](manage-credentials.md#create-a-user-assigned-managed-identity).
89+
*To use a self-hosted integration runtime, you'll first need to [create one](manage-integration-runtimes.md) and confirm your [network settings for Microsoft Purview](catalog-private-link.md)
9090

91-
* **Account Key** - Secrets can be created inside an Azure Key Vault to store credentials in order to enable access for Microsoft Purview to scan data sources securely using the secrets. A secret can be a storage account key, SQL login password, or a password.
92-
93-
> [!Note]
94-
> If you use this option, you need to deploy an _Azure key vault_ resource in your subscription and assign _Microsoft Purview account’s_ SAMI with required access permission to secrets inside _Azure key vault_.
91+
# [System or user assigned managed identity](#tab/MI)
9592

96-
* **Service Principal** - In this method, you can create a new or use an existing service principal in your Azure Active Directory tenant.
93+
#### Using a system or user assigned managed identity for scanning
9794

98-
### Authentication for a scan
95+
There are two types of managed identity you can use:
9996

100-
# [System or user assigned managed identity](#tab/MI)
97+
* **System-assigned managed identity (Recommended)** - As soon as the Microsoft Purview Account is created, a system-assigned managed identity (SAMI) is created automatically in Azure AD tenant. Depending on the type of resource, specific RBAC role assignments are required for the Microsoft Purview system-assigned managed identity (SAMI) to perform the scans.
10198

102-
#### Using a system or user assigned managed identity for scanning
99+
* **User-assigned managed identity** (preview) - Similar to a system managed identity, a user-assigned managed identity (UAMI) is a credential resource that can be used to allow Microsoft Purview to authenticate against Azure Active Directory. For more information, you can see our [User-assigned managed identity guide](manage-credentials.md#create-a-user-assigned-managed-identity).
103100

104101
It's important to give your Microsoft Purview account or user-assigned managed identity (UAMI) the permission to scan the ADLS Gen2 data source. You can add your Microsoft Purview account's system-assigned managed identity (which has the same name as your Microsoft Purview account) or UAMI at the Subscription, Resource Group, or Resource level, depending on what level scan permissions are needed.
105102

@@ -141,6 +138,9 @@ It's important to give your Microsoft Purview account or user-assigned managed i
141138

142139
#### Using Account Key for scanning
143140

141+
> [!Note]
142+
> If you use this option, you need to deploy an _Azure key vault_ resource in your subscription and [assign _Microsoft Purview account’s_ System Assigned Managed Identity (SAMI) required access permission to secrets inside _Azure key vault_.](manage-credentials.md#microsoft-purview-permissions-on-the-azure-key-vault)
143+
144144
When authentication method selected is **Account Key**, you need to get your access key and store in the key vault:
145145

146146
1. Navigate to your ADLS Gen2 storage account
@@ -225,15 +225,19 @@ It's important to give your service principal the permission to scan the ADLS Ge
225225

226226
#### If using Account Key
227227

228-
1. Provide a **Name** for the scan, choose the appropriate collection for the scan, and select **Authentication method** as _Account Key_
228+
1. Provide a **Name** for the scan, select the Azure IR or your Self-Hosted IR depending on your configuration, choose the appropriate collection for the scan, and select **+ New** under credential.
229+
230+
1. Select **Account Key** as the authentication method, then select the appropriate **Key vault connection**, and provide the name of the secret you used to store the account key. Then select **Create**
229231

230232
:::image type="content" source="media/register-scan-adls-gen2/register-adls-gen2-acct-key.png" alt-text="Screenshot that shows the Account Key option for scanning":::
231233

234+
1. Select **Test connection**. On a successful connection, select **Continue**
235+
232236
# [Service Principal](#tab/SP)
233237

234238
#### If using Service Principal
235239

236-
1. Provide a **Name** for the scan, choose the appropriate collection for the scan, and select the **+ New** under **Credential**
240+
1. Provide a **Name** for the scan, select the Azure IR or your Self-Hosted IR depending on your configuration, choose the appropriate collection for the scan, and select the **+ New** under **Credential**
237241

238242
:::image type="content" source="media/register-scan-adls-gen2/register-adls-gen2-sp-option.png" alt-text="Screenshot that shows the option for service principal to enable scanning":::
239243

articles/purview/register-scan-azure-blob-storage-source.md

Lines changed: 23 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -25,23 +25,23 @@ For file types such as csv, tsv, psv, ssv, the schema is extracted when the foll
2525

2626
* First row values are non-empty
2727
* First row values are unique
28-
* First row values are not a date or a number
28+
* First row values aren't a date or a number
2929

3030
## Prerequisites
3131

3232
* An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).
3333

3434
* An active [Microsoft Purview account](create-catalog-portal.md).
3535

36-
* You will need to be a Data Source Administrator and Data Reader to register a source and manage it in the Microsoft Purview governance portal. See our [Microsoft Purview Permissions page](catalog-permissions.md) for details.
36+
* You'll need to be a Data Source Administrator and Data Reader to register a source and manage it in the Microsoft Purview governance portal. See our [Microsoft Purview Permissions page](catalog-permissions.md) for details.
3737

3838
\** Lineage is supported if dataset is used as a source/sink in [Data Factory Copy activity](how-to-link-azure-data-factory.md)
3939

4040
## Register
4141
This section will enable you to register the Azure Blob storage account for scan and data share in Purview.
4242

4343
### Prerequisites for register
44-
* You will need to be a Data Source Admin and one of the other Purview roles (e.g. Data Reader or Data Share Contributor) to register a source and manage it in the Microsoft Purview governance portal. See our [Microsoft Purview Permissions page](catalog-permissions.md) for details.
44+
* You'll need to be a Data Source Admin and one of the other Purview roles (for example, Data Reader or Data Share Contributor) to register a source and manage it in the Microsoft Purview governance portal. See our [Microsoft Purview Permissions page](catalog-permissions.md) for details.
4545

4646
### Steps to register
4747

@@ -81,31 +81,28 @@ For file types such as csv, tsv, psv, ssv, the schema is extracted when the foll
8181

8282
* First row values are non-empty
8383
* First row values are unique
84-
* First row values are neither a date nor a number
84+
* First row values are not a date or a number
8585

8686
### Authentication for a scan
8787

88-
In order to have access to scan the data source, an authentication method in the Azure Blob Storage account needs to be configured.
89-
90-
The following options are supported:
91-
92-
> [!Note]
93-
> If you have firewall enabled for the storage account, you must use managed identity authentication method when setting up a scan.
94-
95-
- **System-assigned managed identity (Recommended)** - As soon as the Microsoft Purview Account is created, a system-assigned managed identity (SAMI) is created automatically in Azure AD tenant. Depending on the type of resource, specific RBAC role assignments are required for the Microsoft Purview SAMI to perform the scans.
88+
Your Azure network may allow for communications between your Azure resources, but if you've set up firewalls, private endpoints, or virtual networks within Azure, you'll need to follow one of these configurations below.
9689

97-
- **User-assigned managed identity** (preview) - Similar to a system-managed identity, a user-assigned managed identity (UAMI) is a credential resource that can be used to allow Microsoft Purview to authenticate against Azure Active Directory. For more information, you can see our [User-assigned managed identity guide](manage-credentials.md#create-a-user-assigned-managed-identity).
90+
|Networking constraints |Integration runtime type |Available credential types |
91+
|---------|---------|---------|
92+
|No private endpoints or firewalls | Azure IR | Managed identity (Recommended), service principal, or account key|
93+
|Firewall enabled but no private endpoints| Azure IR | Managed identity |
94+
|Private endpoints enabled | *Self-Hosted IR | Service principal, account key|
9895

99-
- **Account Key** - Secrets can be created inside an Azure Key Vault to store credentials in order to enable access for Microsoft Purview to scan data sources securely using the secrets. A secret can be a storage account key, SQL login password, or a password.
96+
*To use a self-hosted integration runtime, you'll first need to [create one](manage-integration-runtimes.md) and confirm your [network settings for Microsoft Purview](catalog-private-link.md)
10097

101-
> [!Note]
102-
> If you use this option, you need to deploy an _Azure key vault_ resource in your subscription and assign _Microsoft Purview account’s_ SAMI with required access permission to secrets inside _Azure key vault_.
98+
#### Using a system or user assigned managed identity for scanning
10399

104-
- **Service Principal** - In this method, you can create a new or use an existing service principal in your Azure Active Directory tenant.
100+
There are two types of managed identity you can use:
105101

106-
#### Using a system or user assigned managed identity for scanning
102+
* **System-assigned managed identity (Recommended)** - As soon as the Microsoft Purview Account is created, a system-assigned managed identity (SAMI) is created automatically in Azure AD tenant. Depending on the type of resource, specific RBAC role assignments are required for the Microsoft Purview system-assigned managed identity (SAMI) to perform the scans.
107103

108-
It is important to give your Microsoft Purview account the permission to scan the Azure Blob data source. You can add access for the SAMI or UAMI at the Subscription, Resource Group, or Resource level, depending on what level scan permission is needed.
104+
* **User-assigned managed identity** (preview) - Similar to a system managed identity, a user-assigned managed identity (UAMI) is a credential resource that can be used to allow Microsoft Purview to authenticate against Azure Active Directory. For more information, you can see our [User-assigned managed identity guide](manage-credentials.md#create-a-user-assigned-managed-identity).
105+
It's important to give your Microsoft Purview account the permission to scan the Azure Blob data source. You can add access for the SAMI or UAMI at the Subscription, Resource Group, or Resource level, depending on what level scan permission is needed.
109106

110107
> [!NOTE]
111108
> If you have firewall enabled for the storage account, you must use **managed identity** authentication method when setting up a scan.
@@ -164,14 +161,14 @@ When authentication method selected is **Account Key**, you need to get your acc
164161

165162
1. Select **Create** to complete
166163

167-
1. If your key vault is not connected to Microsoft Purview yet, you will need to [create a new key vault connection](manage-credentials.md#create-azure-key-vaults-connections-in-your-microsoft-purview-account)
164+
1. If your key vault isn't connected to Microsoft Purview yet, you'll need to [create a new key vault connection](manage-credentials.md#create-azure-key-vaults-connections-in-your-microsoft-purview-account)
168165
1. Finally, [create a new credential](manage-credentials.md#create-a-new-credential) using the key to set up your scan
169166

170167
#### Using Service Principal for scanning
171168

172169
##### Creating a new service principal
173170

174-
If you need to [Create a new service principal](./create-service-principal-azure.md), it is required to register an application in your Azure AD tenant and provide access to Service Principal in your data sources. Your Azure AD Global Administrator or other roles such as Application Administrator can perform this operation.
171+
If you need to [Create a new service principal](./create-service-principal-azure.md), it's required to register an application in your Azure AD tenant and provide access to Service Principal in your data sources. Your Azure AD Global Administrator or other roles such as Application Administrator can perform this operation.
175172

176173
##### Getting the Service Principal's Application ID
177174

@@ -181,7 +178,7 @@ If you need to [Create a new service principal](./create-service-principal-azure
181178

182179
##### Granting the Service Principal access to your Azure Blob account
183180

184-
It is important to give your service principal the permission to scan the Azure Blob data source. You can add access for the service principal at the Subscription, Resource Group, or Resource level, depending on what level scan access is needed.
181+
It's important to give your service principal the permission to scan the Azure Blob data source. You can add access for the service principal at the Subscription, Resource Group, or Resource level, depending on what level scan access is needed.
185182

186183
> [!Note]
187184
> You need to be an owner of the subscription to be able to add a service principal on an Azure resource.
@@ -214,13 +211,13 @@ Provide a **Name** for the scan, select the Microsoft Purview accounts SAMI or U
214211

215212
#### If using Account Key
216213

217-
Provide a **Name** for the scan, choose the appropriate collection for the scan, and select **Authentication method** as _Account Key_ and select **Create**
214+
Provide a **Name** for the scan, select the Azure IR or your Self-Hosted IR depending on your configuration, choose the appropriate collection for the scan, and select **Authentication method** as _Account Key_ and select **Create**
218215

219216
:::image type="content" source="media/register-scan-azure-blob-storage-source/register-blob-acct-key.png" alt-text="Screenshot that shows the Account Key option for scanning":::
220217

221218
#### If using Service Principal
222219

223-
1. Provide a **Name** for the scan, choose the appropriate collection for the scan, and select the **+ New** under **Credential**
220+
1. Provide a **Name** for the scan, select the Azure IR or your Self-Hosted IR depending on your configuration, choose the appropriate collection for the scan, and select the **+ New** under **Credential**
224221

225222
:::image type="content" source="media/register-scan-azure-blob-storage-source/register-blob-sp-option.png" alt-text="Screenshot that shows the option for service principal to enable scanning":::
226223

@@ -318,13 +315,13 @@ To map a storage account asset in a received share, you need ONE of the followin
318315
* **Microsoft.Storage/storageAccounts/blobServices/containers/write** - This permission is available in the *Contributor*, *Owner*, *Storage Blob Data Contributor* and *Storage Blob Data Owner* role.
319316

320317
### Update shared data in source storage account
321-
Updates you make to shared files or data in the shared folder from source storage account will be made available to recipient in target storage account in near real time. When you delete subfolder or files within the shared folder, they will disappear for recipient. To delete the shared folder, file or parent folders or containers, you need to first revoke access to all your shares from the source storage account.
318+
Updates you make to shared files or data in the shared folder from source storage account will be made available to recipient in target storage account in near real time. When you delete subfolder or files within the shared folder, they'll disappear for recipient. To delete the shared folder, file or parent folders or containers, you need to first revoke access to all your shares from the source storage account.
322319

323320
### Access shared data in target storage account
324321
The target storage account enables recipient to access the shared data read-only in near real time. You can connect analytics tools such as Synapse Workspace and Databricks to the shared data to perform analytics. Cost of accessing the shared data is charged to the target storage account.
325322

326323
### Service limit
327-
Source storage account can support up to 20 targets, and target storage account can support up to 100 sources. If you require an increase in limit, please contact Support.
324+
Source storage account can support up to 20 targets, and target storage account can support up to 100 sources. If you require an increase in limit, contact Support.
328325

329326
## Access policy
330327

0 commit comments

Comments
 (0)