You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Sometimes you need to execute inference having a higher control of what is being written as output of the batch job. Those cases include:
20
+
This guide explains how to create deployments that generate custom outputs and files. Sometimes you need more control over what's written as output from batch inference jobs. These cases include the following situations:
21
21
22
22
> [!div class="checklist"]
23
-
> * You need to control how the predictions are being written in the output. For instance, you want to append the prediction to the original data (if data is tabular).
24
-
> * You need to write your predictions in a different file format from the one supported out-of-the-box by batch deployments.
23
+
> * You need to control how predictions are written in the output. For instance, you want to append the prediction to the original data if the data is tabular.
24
+
> * You need to write your predictions in a different file format than the one supported out-of-the-box by batch deployments.
25
25
> * Your model is a generative model that can't write the output in a tabular format. For instance, models that produce images as outputs.
26
-
> * Your model produces multiple tabular files instead of a single one. This is the case for instance of models that perform forecasting considering multiple scenarios.
26
+
> * Your model produces multiple tabular files instead of a single one. For example, models that perform forecasting by considering multiple scenarios.
27
27
28
-
In any of those cases, Batch Deployments allow you to take control of the output of the jobs by allowing you to write directly to the output of the batch deployment job. In this tutorial, we'll see how to deploy a model to perform batch inference and writes the outputs in `parquet` format by appending the predictions to the original input data.
28
+
Batch deployments allow you to take control of the output of the jobs by letting you write directly to the output of the batch deployment job. In this tutorial, you learn how to deploy a model to perform batch inference and write the outputs in *parquet* format by appending the predictions to the original input data.
29
29
30
30
## About this sample
31
31
32
-
This example shows how you can deploy a model to perform batch inference and customize how your predictions are written in the output. This example uses a model based on the [UCI Heart Disease Data Set](https://archive.ics.uci.edu/ml/datasets/Heart+Disease). The database contains 76 attributes, but we are using a subset of 14 of them. The model tries to predict the presence of heart disease in a patient. It is integer valued from 0 (no presence) to 1 (presence).
32
+
This example shows how you can deploy a model to perform batch inference and customize how your predictions are written in the output. The model is based on the [UCI Heart Disease dataset](https://archive.ics.uci.edu/ml/datasets/Heart+Disease). The database contains 76 attributes, but this example uses a subset of 14 of them. The model tries to predict the presence of heart disease in a patient. It's integer valued from 0 (no presence) to 1 (presence).
33
33
34
-
The model has been trained using an `XGBBoost` classifier and all the required preprocessing has been packaged as a `scikit-learn` pipeline, making this model an end-to-end pipeline that goes from raw data to predictions.
34
+
The model was trained using an `XGBBoost` classifier and all the required preprocessing was packaged as a `scikit-learn` pipeline, making this model an end-to-end pipeline that goes from raw data to predictions.
@@ -41,34 +41,35 @@ The files for this example are in:
41
41
cd endpoints/batch/deploy-models/custom-outputs-parquet
42
42
```
43
43
44
-
### Follow along in Jupyter Notebooks
44
+
### Follow along in a Jupyter notebook
45
45
46
-
You can follow along this sample in a Jupyter Notebook. In the cloned repository, open the notebook:[custom-output-batch.ipynb](https://github.com/Azure/azureml-examples/blob/main/sdk/python/endpoints/batch/deploy-models/custom-outputs-parquet/custom-output-batch.ipynb).
46
+
There's a Jupyter notebook that you can use to follow this example. In the cloned repository, open the notebook called[custom-output-batch.ipynb](https://github.com/Azure/azureml-examples/blob/main/sdk/python/endpoints/batch/deploy-models/custom-outputs-parquet/custom-output-batch.ipynb).
## Creating a batch deployment with a custom output
52
+
## Create a batch deployment with a custom output
53
53
54
-
In this example, we are going to create a deployment that can write directly to the output folder of the batch deployment job. The deployment will use this feature to write custom parquet files.
54
+
In this example, you create a deployment that can write directly to the output folder of the batch deployment job. The deployment uses this feature to write custom parquet files.
55
55
56
-
### Registering the model
56
+
### Register the model
57
+
58
+
You can only deploy registered models using a batch endpoint. In this case, you already have a local copy of the model in the repository, so you only need to publish the model to the registry in the workspace. You can skip this step if the model you're trying to deploy is already registered.
57
59
58
-
Batch Endpoint can only deploy registered models. In this case, we already have a local copy of the model in the repository, so we only need to publish the model to the registry in the workspace. You can skip this step if the model you are trying to deploy is already registered.
We need to create a scoring script that can read the input data provided by the batch deployment and return the scores of the model. We are also going to write directly to the output folder of the job. In summary, the proposed scoring script does as follows:
72
+
You need to create a scoring script that can read the input data provided by the batch deployment and return the scores of the model. You're also going to write directly to the output folder of the job. In summary, the proposed scoring script does as follows:
72
73
73
74
1. Reads the input data as CSV files.
74
75
2. Runs an MLflow model `predict` function over the input data.
@@ -81,31 +82,31 @@ __code/batch_driver.py__
81
82
82
83
__Remarks:__
83
84
* Notice how the environment variable `AZUREML_BI_OUTPUT_PATH` is used to get access to the output path of the deployment job.
84
-
* The `init()` function is populating a global variable called `output_path` that can be used later to know where to write.
85
-
* The `run` method returns a list of the processed files. It is required for the `run` function to return a `list` or a `pandas.DataFrame` object.
85
+
* The `init()` function populates a global variable called `output_path` that can be used later to know where to write.
86
+
* The `run` method returns a list of the processed files. It's required for the `run` function to return a `list` or a `pandas.DataFrame` object.
86
87
87
88
> [!WARNING]
88
-
> Take into account that all the batch executors will have write access to this path at the same time. This means that you need to account for concurrency. In this case, we are ensuring each executor writes its own file by using the input file name as the name of the output folder.
89
+
> Take into account that all the batch executors have write access to this path at the same time. This means that you need to account for concurrency. In this case, ensure that each executor writes its own file by using the input file name as the name of the output folder.
89
90
90
-
## Creating the endpoint
91
+
## Create the endpoint
91
92
92
-
We are going to create a batch endpoint named `heart-classifier-batch` where to deploy the model.
93
+
You now create a batch endpoint named `heart-classifier-batch` where the model is deployed.
93
94
94
-
1. Decide on the name of the endpoint. The name of the endpoint will end-up in the URI associated with your endpoint. Because of that, __batch endpoint names need to be unique within an Azure region__. For example, there can be only one batch endpoint with the name `mybatchendpoint` in `westus2`.
95
+
1. Decide on the name of the endpoint. The name of the endpoint appears in the URI associated with your endpoint, so *batch endpoint names need to be unique within an Azure region*. For example, there can be only one batch endpoint with the name `mybatchendpoint` in `westus2`.
95
96
96
97
# [Azure CLI](#tab/cli)
97
98
98
-
In this case, let's place the name of the endpoint in a variable so we can easily reference it later.
99
+
In this case, place the name of the endpoint in a variable so you can easily reference it later.
2. Create the deployment. Notice that now `output_action` is set to `SUMMARY_ONLY`.
151
+
2. Create the deployment. Notice that `output_action` is now set to `SUMMARY_ONLY`.
151
152
152
153
> [!NOTE]
153
-
> This example assumes you have aa compute cluster with name `batch-cluster`. Change that name accordinly.
154
+
> This example assumes you have a compute cluster with name `batch-cluster`. Change that name accordingly.
154
155
155
156
# [Azure CLI](#tab/cli)
156
157
157
-
To create a new deployment under the created endpoint, create a `YAML` configuration like the following. You can check the [full batch endpoint YAML schema](reference-yaml-endpoint-batch.md) for extra properties.
158
+
To create a new deployment under the created endpoint, create a YAML configuration like the following. You can check the [full batch endpoint YAML schema](reference-yaml-endpoint-batch.md) for extra properties.
@@ -174,18 +175,18 @@ Follow the next steps to create a deployment using the previous scoring script:
174
175
175
176
3. At this point, our batch endpoint is ready to be used.
176
177
177
-
## Testing out the deployment
178
+
## Test the deployment
178
179
179
-
For testing our endpoint, we are going to use a sample of unlabeled data located in this repository and that can be used with the model. Batch endpoints can only process data that is located in the cloud and that is accessible from the Azure Machine Learning workspace. In this example, we are going to upload it to an Azure Machine Learning data store. Particularly, we are going to create a data asset that can be used to invoke the endpoint for scoring. However, notice that batch endpoints accept data that can be placed in multiple type of locations.
180
+
To test your endpoint, use a sample of unlabeled data located in this repository, which can be used with the model. Batch endpoints can only process data that's located in the cloud and is accessible from the Azure Machine Learning workspace. In this example, you upload it to an Azure Machine Learning data store. You're going to create a data asset that can be used to invoke the endpoint for scoring. However, notice that batch endpoints accept data that can be placed in multiple type of locations.
180
181
181
-
1.Let's invoke the endpoint with data from a storage account:
182
+
1.Invoke the endpoint with data from a storage account:
The job generates a named output called `score` where all the generated files are placed. Since we wrote into the directory directly, one file per each input file, then we can expect to have the same number of files. In this particular example we decided to name the output files the same as the inputs, but they will have a parquet extension.
216
+
The job generates a named output called `score` where all the generated files are placed. Since you wrote into the directory directly, one file per each input file, then you can expect to have the same number of files. In this particular example, name the output files the same as the inputs, but they have a parquet extension.
216
217
217
218
> [!NOTE]
218
-
> Notice that a file `predictions.csv` is also included in the output folder. This file contains the summary of the processed files.
219
+
> Notice that a file *predictions.csv* is also included in the output folder. This file contains the summary of the processed files.
219
220
220
221
You can download the results of the job by using the job name:
221
222
@@ -228,6 +229,7 @@ To download the predictions, use the following command:
Copy file name to clipboardExpand all lines: articles/machine-learning/includes/azureml-batch-prereqs.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,15 +10,15 @@ Before following the steps in this article, make sure you have the following pre
10
10
11
11
* An Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the [free or paid version of Azure Machine Learning](https://azure.microsoft.com/free/).
12
12
13
-
* An Azure Machine Learning workspace. If you don't have one, use the steps in the [How to manage workspaces](../how-to-manage-workspace.md) article to create one.
13
+
* An Azure Machine Learning workspace. If you don't have one, use the steps in the [Manage Azure Machine Learning workspaces](../how-to-manage-workspace.md) article to create one.
14
14
15
-
* Ensure you have the following permissions in the workspace:
15
+
* Ensure that you have the following permissions in the workspace:
16
16
17
-
* Create/manage batch endpoints and deployments: Use roles Owner, contributor, or custom role allowing`Microsoft.MachineLearningServices/workspaces/batchEndpoints/*`.
17
+
* Create or manage batch endpoints and deployments: Use an Owner, Contributor, or Custom role that allows`Microsoft.MachineLearningServices/workspaces/batchEndpoints/*`.
18
18
19
-
* Create ARM deployments in the workspace resource group: Use roles Owner, contributor, or custom role allowing`Microsoft.Resources/deployments/write` in the resource group where the workspace is deployed.
19
+
* Create ARM deployments in the workspace resource group: Use an Owner, Contributor, or Custom role that allows`Microsoft.Resources/deployments/write` in the resource group where the workspace is deployed.
20
20
21
-
* You will need to install the following software to work with Azure Machine Learning:
21
+
* You need to install the following software to work with Azure Machine Learning:
0 commit comments