Skip to content

Commit 2980ddb

Browse files
ttorblettorble
authored
Merge pull request #114443 from Samantha-Yu/adfupdate0508
New article: Data access strategies
2 parents 6f6fff0 + 686b178 commit 2980ddb

File tree

2 files changed

+81
-0
lines changed

2 files changed

+81
-0
lines changed

‎articles/data-factory/TOC.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -543,6 +543,8 @@
543543
items:
544544
- name: Data movement security considerations
545545
href: data-movement-security-considerations.md
546+
- name: Data access strategies
547+
href: data-access-strategies.md
546548
- name: Azure Integration Runtime IP addresses
547549
href: azure-integration-runtime-ip-addresses.md
548550
- name: Store credentials in Azure Key Vault

‎articles/data-factory/data-access-strategies.md

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
---
2+
title: Data access strategies
3+
description: Azure Data Factory now supports Static IP address ranges.
4+
services: data-factory
5+
ms.author: abnarain
6+
author: nabhishek
7+
ms.service: data-factory
8+
ms.workload: data-services
9+
ms.topic: conceptual
10+
ms.date: 05/11/2020
11+
---
12+
13+
# Data access strategies
14+
15+
A vital security goal of an organization is to protect their data stores from random access over the internet, may it be an on-premise or a Cloud/ SaaS data store.
16+
17+
Typically a cloud data store controls access using the below mechanisms:
18+
* Firewall rules that limit connectivity by IP address
19+
* Authentication mechanisms that require users to prove their identity
20+
* Authorization mechanisms that restrict users to specific actions and data
21+
22+
> [!TIP]
23+
> With the [introduction of Static IP address range](https://docs.microsoft.com/azure/data-factory/azure-integration-runtime-ip-addresses), you can now allow list IP ranges for the particular Azure integration runtime region to ensure you don’t have to allow all Azure IP addresses in your cloud data stores. This way, you can restrict the IP addresses that are permitted to access the data stores.
24+
25+
> [!NOTE]
26+
> The IP address ranges are blocked for Azure integration runtime and is currently only used for Data Movement, pipeline and external activities. Dataflows now do not use these IP ranges.
27+
28+
This should work in many scenarios, and we do understand that a unique Static IP address per integration runtime would be desirable, but this wouldn't be possible using Azure Integration Runtime currently, which is serverless. If necessary, you can always set up a Self-hosted Integration Runtime and use your Static IP with it.
29+
30+
## Data access strategies through Azure Data Factory
31+
32+
* **[Trusted Service](https://docs.microsoft.com/azure/storage/common/storage-network-security#exceptions)** - Azure Storage (Blob, ADLS Gen2) supports firewall configuration that enables select trusted Azure platform services to access the storage account securely. Trusted Services enforces Managed Identity authentication, which ensures no other data factory can connect to this storage unless whitelisted to do so using it's managed identity. You can find more details in **[this blog](https://techcommunity.microsoft.com/t5/azure-data-factory/data-factory-is-now-a-trusted-service-in-azure-storage-and-azure/ba-p/964993)**. Hence, this is extremely secure and recommended.
33+
* **Unique Static IP** - You will need to set up a self-hosted integration runtime to get a Static IP for Data Factory connectors. This mechanism ensures you can block access from all other IP addresses.
34+
* **[Static IP range](https://docs.microsoft.com/azure/data-factory/azure-integration-runtime-ip-addresses)** - You can use Azure Integration Runtime's IP addresses to allow list it in your storage (say S3, Salesforce, etc.). It certainly restricts IP addresses that can connect to the data stores but also relies on Authentication/ Authorization rules.
35+
* **[Service Tag](https://docs.microsoft.com/azure/virtual-network/service-tags-overview)** - A service tag represents a group of IP address prefixes from a given Azure service (like Azure Data Factory). Microsoft manages the address prefixes encompassed by the service tag and automatically updates the service tag as addresses change, minimizing the complexity of frequent updates to network security rules. It is useful when whitelisting data access on IaaS hosted data stores in Virtual Network.
36+
* **Allow Azure Services** - Some services lets you allow all Azure services to connect to it in case you choose this option.
37+
38+
For more information about supported network security mechanisms on data stores in Azure Integration Runtime and Self-hosted Integration Runtime, see below two tables.
39+
* **Azure Integration Runtime**
40+
41+
| Data Stores | Supported Network Security Mechanism on Data Stores | Trusted Service | Static IP range | Service Tags | Allow Azure Services |
42+
|------------------------------|-------------------------------------------------------------|---------------------|-----------------|--------------|----------------------|
43+
| Azure PaaS Data stores | Azure Cosmos DB | - | Yes | - | Yes |
44+
| | Azure Data Explorer | - | Yes* | Yes* | - |
45+
| | Azure Data Lake Gen1 | - | Yes | - | Yes |
46+
| | Azure Database for MariaDB, MySQL, PostgreSQL | - | Yes | - | Yes |
47+
| | Azure File Storage | - | Yes | - | . |
48+
| | Azure Storage (Blog, ADLS Gen2) | Yes (MSI auth only) | Yes | - | . |
49+
| | Azure SQL DB, SQL DW (Synapse Analytics), SQL Ml | - | Yes | - | Yes |
50+
| | Azure Key Vault (for fetching secrets/ connection string) | Yes | Yes | - | - |
51+
| Other PaaS/ SaaS Data stores | AWS S3, SalesForce, Google Cloud Storage, etc. | - | Yes | - | - |
52+
| Azure laaS | SQL Server, Oracle, etc. | - | Yes | Yes | - |
53+
| On-premise laaS | SQL Server, Oracle, etc. | - | Yes | - | - |
54+
55+
**Applicable only when Azure Data Explorer is virtual network injected, and IP range can be applied on NSG/ Firewall.*
56+
57+
* **Self-hosted Integration Runtime (in Vnet/on-premise)**
58+
59+
| Data Stores | Supported Network Security Mechanism on Data Stores | Static IP | Trusted Services |
60+
|--------------------------------|---------------------------------------------------------------|-----------|---------------------|
61+
| Azure PaaS Data stores | Azure Cosmos DB | Yes | - |
62+
| | Azure Data Explorer | - | - |
63+
| | Azure Data Lake Gen1 | Yes | - |
64+
| | Azure Database for MariaDB, MySQL, PostgreSQL | Yes | - |
65+
| | Azure File Storage | Yes | - |
66+
| | Azure Storage (Blog, ADLS Gen2) | Yes | Yes (MSI auth only) |
67+
| | Azure SQL DB, SQL DW (Synapse Analytics), SQL Ml | Yes | - |
68+
| | Azure Key Vault (for fetching secrets/ connection string) | Yes | Yes |
69+
| Other PaaS/ SaaS Data stores | AWS S3, SalesForce, Google Cloud Storage, etc. | Yes | - |
70+
| Azure laaS | SQL Server, Oracle, etc. | Yes | - |
71+
| On-premise laaS | SQL Server, Oracle, etc. | Yes | - |
72+
73+
## Next steps
74+
75+
For more information, see the following related articles:
76+
* [Supported data stores](https://docs.microsoft.com/azure/data-factory/copy-activity-overview#supported-data-stores-and-formats)
77+
* [Azure Key Vault ‘Trusted Services’](https://docs.microsoft.com/azure/key-vault/key-vault-overview-vnet-service-endpoints#trusted-services)
78+
* [Azure Storage ‘Trusted Microsoft Services’](https://docs.microsoft.com/azure/storage/common/storage-network-security#trusted-microsoft-services)
79+
* [Managed identity for Data Factory](https://docs.microsoft.com/azure/data-factory/data-factory-service-identity)

0 commit comments

Comments
 (0)