You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-use-batch-model-deployments.md
+3-44Lines changed: 3 additions & 44 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -34,31 +34,6 @@ Use batch endpoints for model deployment when:
34
34
35
35
In this article, you use a batch endpoint to deploy a machine learning model that solves the classic MNIST (Modified National Institute of Standards and Technology) digit recognition problem. Your deployed model then performs batch inferencing over large amounts of data—in this case, image files. You begin by creating a batch deployment of a model that was created using Torch. This deployment becomes the default one in the endpoint. Later, you [create a second deployment](#add-deployments-to-an-endpoint) of a mode that was created with TensorFlow (Keras), test the second deployment, and then set it as the endpoint's default deployment.
36
36
37
-
## Key concepts and configurations
38
-
39
-
Review the sections below to understand the key concepts and configurations used in this article.
40
-
41
-
### Data flow overview
42
-
-__Input data__: Files or folders in Azure Storage (blob storage, data lake, or registered datasets)
43
-
-__Processing__: Your model processes data in configurable mini-batches
|`mini_batch_size`| Files per batch (affects memory usage) | 10-50 for large files, 100-500 for small files |
51
-
|`instance_count`| Number of compute instances | 1-10 depending on data volume |
52
-
|`max_concurrency_per_instance`| Parallel processes per instance | 1 for memory-intensive models, 2-4 for lightweight models |
53
-
|`timeout`| Max time per mini-batch (seconds) | 30-300 depending on model complexity |
54
-
|`output_action`| How to organize results |`append_row` (combine all results) or `summary_only`|
55
-
56
-
### Common parameters for job invocation
57
-
-__Azure CLI__: Use `--input`, `--output-path`, `--set` for overrides
58
-
-__Python SDK__: Use `Input()` for data, `params_override` for settings
59
-
-__Studio__: Use the web interface to configure inputs, outputs, and deployment settings
60
-
61
-
To follow along with the code samples and files needed to run the commands in this article locally, see the __[Clone the examples repository](#clone-the-examples-repository)__ section. The code samples and files are contained in the [azureml-examples](https://github.com/azure/azureml-examples) repository.
62
37
63
38
## Prerequisites
64
39
@@ -611,7 +586,9 @@ Use `output-path` to configure any folder in an Azure Machine Learning registere
611
586
612
587
# [Python](#tab/python)
613
588
614
-
**Understanding `params_override`**: The `params_override` parameter allows you to modify deployment settings for a specific job without changing the deployment configuration permanently. This is useful for adjusting settings like output location, mini-batch size, or instance count for individual jobs.
589
+
The `params_override` parameter allows you to modify deployment settings for a specific job without changing the deployment configuration permanently. This is useful for adjusting settings like output location, mini-batch size, or instance count for individual jobs.
590
+
591
+
The `params_override` parameter values correspond to deployment configuration settings that can be temporarily modified for individual jobs. These parameters come from your deployment's YAML schema settings, datastore configurations (like output paths), and runtime variables you define in your code.
615
592
616
593
Use `params_override` to configure any folder in an Azure Machine Learning registered data store. Only registered data stores are supported as output paths. In this example you use the default data store:
617
594
@@ -621,24 +598,6 @@ Once you've identified the data store you want to use, configure the output as f
The `params_override` parameter values correspond to deployment configuration settings that can be temporarily modified for individual jobs. These parameters come from your deployment's YAML schema settings, datastore configurations (like output paths), and runtime variables you define in your code.
627
-
628
-
```python
629
-
# Override multiple settings for this specific job
> Use `params_override` when you need different settings for different jobs without modifying your deployment. This is especially useful for handling varying data sizes or experimenting with performance settings.
0 commit comments