You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Tutorial: Use Batch Explorer, Storage Explorer, and Python to run a Batch job through Data Factory
11
11
12
-
This tutorial walks you through creating and running an Azure Data Factory pipeline that runs an Azure Batch data manipulation workload. A Python script runs on the Batch nodes to get comma-separated value (CSV) input from an Azure Blob Storage container, manipulate the data, and write the output to a different storage container. You use Batch Explorer to create a Batch pool and nodes, and Azure Storage Explorer to work with storage containers and files.
12
+
This tutorial walks you through creating and running an Azure Data Factory pipeline that runs an Azure Batch workload. A Python script runs on the Batch nodes to get comma-separated value (CSV) input from an Azure Blob Storage container, manipulate the data, and write the output to a different storage container. You use Batch Explorer to create a Batch pool and nodes, and Azure Storage Explorer to work with storage containers and files.
13
13
14
14
In this tutorial, you learn how to:
15
15
16
16
> [!div class="checklist"]
17
17
> - Use Batch Explorer to create a Batch pool and nodes.
18
18
> - Use Storage Explorer to create storage containers and upload files.
19
-
> - Develop a Python script that runs a Batch workload.
19
+
> - Develop a Python script that does a Batch data manipulation task.
20
20
> - Create a Data Factory pipeline that runs the Batch workload.
21
21
> - Use Batch Explorer to look at the output log files.
22
22
@@ -62,8 +62,7 @@ Use Storage Explorer to create blob containers to store input and output files,
62
62
1. Create another blob container named *output*.
63
63
1. Select the **input** container, and then select **Upload** > **Upload files** in the right pane.
64
64
1. On the **Upload files** screen, under **Selected files**, select the ellipsis **...** next to the entry field.
65
-
1. Browse to the location of your downloaded *iris.csv* file, and select **Open**.
66
-
1. Select **Upload**.
65
+
1. Browse to the location of your downloaded *iris.csv* file, select **Open**, and then select **Upload**.
67
66
68
67
[](media/run-python-batch-azure-data-factory/storage-explorer.png#lightbox)
69
68
## Develop a Python script
@@ -76,7 +75,7 @@ The script needs to use the connection string for the Azure Storage account that
76
75
1. On the page for the storage account, select **Access keys** from the left navigation under **Security + networking**.
77
76
1. Under **key1**, select **Show** next to **Connection string**, and then select the **Copy** icon to copy the connection string.
78
77
79
-
Paste the connection string into the following script to replace the `<storage-account-connection-string>` placeholder. Save the script as a file named *main.py*.
78
+
Paste the connection string into the following script, replacing the `<storage-account-connection-string>` placeholder. Save the script as a file named *main.py*.
80
79
81
80
>[!IMPORTANT]
82
81
>Exposing account keys in the app source isn't recommended for Production usage. You should restrict access to credentials and refer to them in your code by using variables or a configuration file. It's best to store Batch and Storage account keys in Azure Key Vault.
0 commit comments