You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/data-factory/airflow-configurations.md
+7-4Lines changed: 7 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
---
2
-
title: Apache Airflow configuration options on Managed Airflow
3
-
description: This article explains how Apache Airflow configuration options can be attached to your Managed Airflow integration runtimes for an Apache Airflow environment as key-value pairs.
2
+
title: Supported Apache Airflow configurations
3
+
description: This article enumerates the Apache Airflow configurations supported by the Workflow Orchestration Manager.
4
4
author: nabhishek
5
5
ms.author: abnarain
6
6
ms.reviewer: jburchel
@@ -9,9 +9,12 @@ ms.topic: how-to
9
9
ms.date: 12/11/2023
10
10
---
11
11
12
-
# Apache Airflow configuration options on Managed Airflow
12
+
# Supported Apache Airflow configurations
13
13
14
-
Apache Airflow configuration options can be attached to your Azure Data Factory Managed Airflow integration runtime as key-value pairs. We don't expose the `airflow.cfg` in the Managed Airflow UI. However, users can override the Apache Airflow configurations directly on the UI as key-value pairs under the `Airflow Configuration overrides` section. They can continue using all other settings in `airflow.cfg`. In Managed Airflow, developers can override any of the Airflow configurations provided by Apache Airflow except the configurations shown in the table.
14
+
> [!NOTE]
15
+
> This feature is in public preview. Workflow Orchestration Manager is powered by Apache Airflow.
16
+
17
+
In Workflow Orchestration Manager, Apache Airflow configurations can be integrated with the platform's runtime as key-value pairs. While the `airflow.cfg` isn't directly accessible in the UI, users can override these configurations via the UI's "Airflow Configuration overrides" section, retaining access to other `airflow.cfg` settings. Developers have the flexibility to override most Apache Airflow configurations within Workflow Orchestration Manager, `except for those explicitly outlined in a provided table`.
This article explains how to enhance security of your data stores and resources by restricting access solely to your Azure Data Factory Managed Airflow cluster. In this article, you walk through the process of retrieving and adding the unique IP address associated with your Managed Airflow cluster to your storage firewall's allowlist. This process enables you to access data stores or resources through the list of permitted IP addresses on the firewall's allowlist. Access from all other IP addresses via the public endpoint is prevented.
16
+
> [!NOTE]
17
+
> This feature is in public preview. Workflow Orchestration Manager is powered by Apache Airflow.
18
+
19
+
This article demonstrates how to retrieve and add the IP address associated with your Workflow Orchestration Manager cluster to your storage firewall's allowlist. This enhances the security of data stores and resources by restricting access solely to the Workflow Orchestration Manager cluster within Azure Data Factory, preventing access from all other IP addresses via the public endpoint.
17
20
18
21
> [!NOTE]
19
22
> Importing DAGs is currently not supported by using blob storage with IP allow listing or by using private endpoints. We suggest using Git sync instead.
@@ -37,33 +40,33 @@ For more information, see the following screenshots.
37
40
38
41
:::image type="content" source="media/airflow-get-ip-airflow-cluster/get-dags.png" alt-text="Screenshot that shows a sample Airflow API request using a bearer token fetched in the initial step." lightbox="media/airflow-get-ip-airflow-cluster/get-dags.png":::
39
42
40
-
### Retrieve the Managed Airflow cluster's IP address
43
+
### Retrieve the Workflow Orchestration Manager cluster's IP address
41
44
42
-
1. Use the Managed Airflow UI.
45
+
1. Use the Workflow Orchestration Manager UI.
43
46
44
47
:::image type="content" source="media/airflow-get-ip-airflow-cluster/get-cluster-ip-from-ui.png" alt-text="Screenshot that shows how to retrieve a cluster's IP by using the UI." lightbox="media/airflow-get-ip-airflow-cluster/get-cluster-ip-from-ui.png":::
45
48
46
49
1. Use the Rest API.
47
-
For more information, see [Managed Airflow IP address - Get](/rest/api/datafactory/integration-runtimes/get?tabs=HTTP#code-try-0).
50
+
For more information, see [Workflow Orchestration Manager IP address - Get](/rest/api/datafactory/integration-runtimes/get?tabs=HTTP#code-try-0).
48
51
49
52
You should retrieve the Airflow cluster's IP address from the response, as shown in the screenshot.
50
53
51
54
#### Sample response
52
55
53
56
:::image type="content" source="media/airflow-get-ip-airflow-cluster/get-cluster-ip-from-api.png" alt-text="Screenshot that shows how to retrieve a cluster's IP by using an API." lightbox="media/airflow-get-ip-airflow-cluster/get-cluster-ip-from-api.png":::
54
57
55
-
### Add the Managed Airflow cluster IP address to the storage account you want to secure
58
+
### Add the Workflow Orchestration Manager cluster IP address to the storage account you want to secure
56
59
57
60
> [!NOTE]
58
-
> You can add the Managed Airflow IP address to other storage services too, like Azure SQL Database and Azure Key Vault.
61
+
> You can add the Workflow Orchestration Manager IP address to other storage services too, like Azure SQL Database and Azure Key Vault.
59
62
60
-
- To add a Managed Airflow cluster IP address in Azure Key Vault, see [Azure SQL Database and Azure Synapse IP firewall rules](/azure/key-vault/general/network-security).
61
-
- To add a Managed Airflow cluster IP address in Azure Blob Storage, see [Configure Azure Storage firewalls and virtual networks](/azure/storage/common/storage-network-security?tabs=azure-portal#grant-access-from-an-internet-ip-range).
62
-
- To add a Managed Airflow cluster IP address in Azure SQL Database, see [Configure Azure Key Vault firewalls and virtual networks](/azure/azure-sql/database/firewall-configure).
63
-
- To add a Managed Airflow cluster IP address in Azure PostgreSQL Database, see [Create and manage firewall rules for Azure Database for PostgreSQL - Single server using the Azure portal](/azure/postgresql/single-server/how-to-manage-firewall-using-portal).
63
+
- To add a Workflow Orchestration Manager cluster IP address in Azure Key Vault, see [Azure SQL Database and Azure Synapse IP firewall rules](/azure/key-vault/general/network-security).
64
+
- To add a Workflow Orchestration Manager cluster IP address in Azure Blob Storage, see [Configure Azure Storage firewalls and virtual networks](/azure/storage/common/storage-network-security?tabs=azure-portal#grant-access-from-an-internet-ip-range).
65
+
- To add a Workflow Orchestration Manager cluster IP address in Azure SQL Database, see [Configure Azure Key Vault firewalls and virtual networks](/azure/azure-sql/database/firewall-configure).
66
+
- To add a Workflow Orchestration Manager cluster IP address in Azure PostgreSQL Database, see [Create and manage firewall rules for Azure Database for PostgreSQL - Single server using the Azure portal](/azure/postgresql/single-server/how-to-manage-firewall-using-portal).
64
67
65
68
## Related content
66
69
67
-
-[Run an existing pipeline with Managed Airflow](tutorial-run-existing-pipeline-with-airflow.md)
68
-
-[Managed Airflow pricing](airflow-pricing.md)
69
-
-[Change the password for Managed Airflow environments](password-change-airflow.md)
70
+
-[Run an existing pipeline with Workflow Orchestration Manager](tutorial-run-existing-pipeline-with-airflow.md)
0 commit comments