You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-nlp-processing-batch.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,7 +21,7 @@ Batch Endpoints can be used for processing tabular data, but also any other file
21
21
22
22
## About this sample
23
23
24
-
The model we are going to work with was built using the popular library transformers from HuggingFace along with [a pre-trained model from Facebook with the BART architecture](https://huggingface.co/facebook/bart-large-cnn). It was introduced in the paper [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation](https://arxiv.org/abs/1910.13461). This model has the following constrains which are important to keep in mind for deployment:
24
+
The model we are going to work with was built using the popular library transformers from HuggingFace along with [a pre-trained model from Facebook with the BART architecture](https://huggingface.co/facebook/bart-large-cnn). It was introduced in the paper [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation](https://arxiv.org/abs/1910.13461). This model has the following constraints which are important to keep in mind for deployment:
25
25
26
26
* It can work with sequences up to 1024 tokens.
27
27
* It is trained for summarization of text in English.
@@ -90,7 +90,7 @@ model_local_path = 'model'
90
90
summarizer.save_pretrained(model_local_path)
91
91
```
92
92
93
-
We can now register this model in the Azure Machine Leanring registry:
93
+
We can now register this model in the Azure Machine Learning registry:
94
94
95
95
# [Azure CLI](#tab/cli)
96
96
@@ -135,7 +135,7 @@ We are going to create a batch endpoint named `text-summarization-batch` where t
135
135
136
136
# [Azure CLI](#tab/azure-cli)
137
137
138
-
The following YAML file defines a batch endpoint:.
138
+
The following YAML file defines a batch endpoint:
139
139
140
140
__endpoint.yml__
141
141
@@ -183,7 +183,7 @@ Let's create the deployment that will host the model:
183
183
> [!TIP]
184
184
> Although files are provided in mini-batches by the deployment, this scoring script processes one row at a time. This is a common pattern when dealing with expensive models (like transformers) as trying to load the entire batch and send it to the model at once may result in high-memory pressure on the batch executor (OOM exeptions).
185
185
186
-
1. We need to indicate over which environment we are going to run the deployment. In our case, our model runs on `Torch` and it requires the libraries `transformers`, `accelerate`, and `optimium` from HuggingFace. Azure Machine Learning already has an environment with Torch and GPU support available. We are just going to add a couple of dependencies in a `conda.yml` file.
186
+
1. We need to indicate over which environment we are going to run the deployment. In our case, our model runs on `Torch` and it requires the libraries `transformers`, `accelerate`, and `optimum` from HuggingFace. Azure Machine Learning already has an environment with Torch and GPU support available. We are just going to add a couple of dependencies in a `conda.yml` file.
187
187
188
188
__environment/conda.yml__
189
189
@@ -239,7 +239,7 @@ Let's create the deployment that will host the model:
239
239
> [!NOTE]
240
240
> You are not charged for compute at this point as the cluster will remain at 0 nodes until a batch endpoint is invoked and a batch scoring job is submitted. Learn more about [manage and optimize cost for AmlCompute](./how-to-manage-optimize-cost.md#use-azure-machine-learning-compute-cluster-amlcompute).
241
241
242
-
1. Now, let create the deployment.
242
+
1. Now, let's create the deployment.
243
243
244
244
# [Azure CLI](#tab/cli)
245
245
@@ -374,7 +374,7 @@ As mentioned in some of the notes along this tutorial, processing text may have
374
374
375
375
## Considerations for MLflow models that process text
376
376
377
-
The same considerations mentioned above apply to MLflow models. However, since you are not required to provide a scoring script for your MLflow model deployment, some of the recommendation mentioned may require a different approach.
377
+
The same considerations mentioned above apply to MLflow models. However, since you are not required to provide a scoring script for your MLflow model deployment, some of the recommendations mentioned may require a different approach.
378
378
379
379
* MLflow models in Batch Endpoints support reading tabular data as input data, which may contain long sequences of text. See [File's types support](how-to-mlflow-batch.md#files-types-support) for details about which file types are supported.
380
380
* Batch deployments will call your MLflow model's predict function with the content of an entire file in as Pandas dataframe. If your input data contains many rows, chances are that running a complex model (like the one presented in this tutorial) will result in an out-of-memory exception. If this is your case, you can consider:
0 commit comments