You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/data-factory/how-does-managed-airflow-work.md
+11-11Lines changed: 11 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,7 +19,7 @@ ms.date: 01/20/2023
19
19
Azure Data Factory Managed Airflow orchestrates your workflows using Directed Acyclic Graphs (DAGs) written in Python. You must provide your DAGs and plugins in Azure Blob Storage. Airflow requirements or library dependencies can be installed during the creation of the new Managed Airflow environment or by editing an existing Managed Airflow environment. Then run and monitor your DAGs by launching the Airflow UI from ADF using a command line interface (CLI) or a software development kit (SDK).
20
20
21
21
## Create a Managed Airflow environment
22
-
The following steps setup and configure your Managed Airflow environment.
22
+
The following steps set up and configure your Managed Airflow environment.
23
23
24
24
### Prerequisites
25
25
**Azure subscription**: If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/) before you begin.
@@ -46,7 +46,7 @@ The following steps describe how to import DAGs into Managed Airflow.
46
46
47
47
### Prerequisites
48
48
49
-
You will need to upload a sample DAG onto an accessible Storage account.
49
+
You'll need to upload a sample DAG onto an accessible Storage account.
50
50
51
51
> [!NOTE]
52
52
> Blob Storage behind VNet are not supported during the preview.
@@ -63,15 +63,15 @@ You will need to upload a sample DAG onto an accessible Storage account.
63
63
> [!NOTE]
64
64
> You will need to select a directory path from a blob storage account that contains folders named **dags** and **plugins** to import those into the Airflow environment. **Plugins** are not mandatory. You can also have a container named **dags** and upload all Airflow files within it.
65
65
66
-
1.Click on **Airflow (Preview)** under **Manage** hub. Then hover over the earlier created **Airflow** environment and click on **Import files** to Import all DAGs and dependencies into the Airflow Environment.
66
+
1.Select on **Airflow (Preview)** under **Manage** hub. Then hover over the earlier created **Airflow** environment and select on **Import files** to Import all DAGs and dependencies into the Airflow Environment.
67
67
68
68
:::image type="content" source="media/how-does-managed-airflow-work/import-files.png" alt-text="Screenshot shows import files in manage hub.":::
69
69
70
70
1. Create a new Linked Service to the accessible storage account mentioned in the prerequisite (or use an existing one if you already have your own DAGs).
71
71
72
72
:::image type="content" source="media/how-does-managed-airflow-work/create-new-linked-service.png" alt-text="Screenshot that shows how to create a new linked service.":::
73
73
74
-
1. Use the storage account where you uploaded the DAG (check prerequisite). Test connection, then click**Create**.
74
+
1. Use the storage account where you uploaded the DAG (check prerequisite). Test connection, then select**Create**.
75
75
76
76
:::image type="content" source="media/how-does-managed-airflow-work/linked-service-details.png" alt-text="Screenshot shows some linked service details.":::
77
77
@@ -96,27 +96,27 @@ You will need to upload a sample DAG onto an accessible Storage account.
96
96
* Problem: DAG import is taking over 5 minutes
97
97
Mitigation: Reduce the size of the imported DAGs with a single import. One way to achieve this is by creating multiple DAG folders with lesser DAGs across multiple containers.
98
98
99
-
* Problem: Imported DAGs do not show up when you login into the Airflow UI.
100
-
Mitigation: Login into the Airflow UI and see if there are any DAG parsing errors. This could happen if the DAG files contains any incompatible code. You will find the exact line numbers and the files which have the issue through the Airflow UI.
99
+
* Problem: Imported DAGs don't show up when you sign in into the Airflow UI.
100
+
Mitigation: Sign in into the Airflow UI and see if there are any DAG parsing errors. This could happen if the DAG files contain any incompatible code. You'll find the exact line numbers and the files, which have the issue through the Airflow UI.
101
101
102
102
:::image type="content" source="media/how-does-managed-airflow-work/import-dag-issues.png" alt-text="Screenshot shows import dag issues.":::
103
103
104
104
105
105
## Monitor DAG runs
106
106
107
-
To monitor the Airflow DAGs, login into Airflow UI with the earlier created username and password.
107
+
To monitor the Airflow DAGs, sign in into Airflow UI with the earlier created username and password.
108
108
109
-
1.Click on the Airflow environment created.
109
+
1.Select on the Airflow environment created.
110
110
111
111
:::image type="content" source="media/how-does-managed-airflow-work/airflow-environment-monitor-dag.png" alt-text="Screenshot that shows the Airflow environment created.":::
112
112
113
-
1.Login using the username-password provided during the Airflow Integration Runtime creation. ([You can reset the username or password by editing the Airflow Integration runtime]() if needed)
113
+
1.Sign in using the username-password provided during the Airflow Integration Runtime creation. ([You can reset the username or password by editing the Airflow Integration runtime]() if needed)
114
114
115
-
:::image type="content" source="media/how-does-managed-airflow-work/login-in-dags.png" alt-text="Screenshot that shows login using the username-password provided during the Airflow Integration Runtime creation.":::
115
+
:::image type="content" source="media/how-does-managed-airflow-work/login-in-dags.png" alt-text="Screenshot that shows sign in using the username-password provided during the Airflow Integration Runtime creation.":::
116
116
117
117
## Remove DAGs from the Airflow environment
118
118
119
-
If you are using Airflow version 1.x, delete DAGs that are deployed on any Airflow environment (IR), you need to delete the DAGs in two different places.
119
+
If you're using Airflow version 1.x, delete DAGs that are deployed on any Airflow environment (IR), you need to delete the DAGs in two different places.
0 commit comments