You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/concept-online-deployment-model-specification.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -29,7 +29,7 @@ In Azure Machine Learning, after you create your deployment, the environment var
29
29
30
30
We use the following local folder structure to illustrate how you can specify models that are available locally on your machine in an online deployment.
31
31
32
-
:::image type="content" source="media/how-to-deploy-online-endpoints/multi-models-1.png" alt-text="A screenshot showing a local folder structure containing multiple models." lightbox="media/how-to-deploy-online-endpoints/multi-models-1.png":::
32
+
:::image type="content" source="media/concept-online-deployment-model-specification/multi-models-1.png" alt-text="A screenshot showing a local folder structure containing multiple models." lightbox="media/concept-online-deployment-model-specification/multi-models-1.png":::
33
33
34
34
### Deployment configuration with a single local model
35
35
@@ -87,7 +87,7 @@ After you create your deployment, the environment variable `AZUREML_MODEL_DIR` p
87
87
88
88
For this example, the contents of the `AZUREML_MODEL_DIR` folder look like this:
89
89
90
-
:::image type="content" source="media/how-to-deploy-online-endpoints/multi-models-2.png" alt-text="A screenshot showing the folder structure of the storage location for multiple models." lightbox="media/how-to-deploy-online-endpoints/multi-models-2.png":::
90
+
:::image type="content" source="media/concept-online-deployment-model-specification/multi-models-2.png" alt-text="A screenshot showing the folder structure of the storage location for multiple models." lightbox="media/concept-online-deployment-model-specification/multi-models-2.png":::
91
91
92
92
Within your scoring script (`score.py`), you can load your models in the `init()` function. The following code loads the `sample_m1.pkl` model:
93
93
@@ -123,11 +123,11 @@ instance_count: 1
123
123
124
124
For this example, consider that `local-multimodel:3` contains the following model artifacts, which can be viewed from the **Models** tab in the Azure Machine Learning studio:
125
125
126
-
:::image type="content" source="media/how-to-deploy-online-endpoints/multi-models-3.png" alt-text="A screenshot of a folder structure showing the model artifacts of a registered model." lightbox="media/how-to-deploy-online-endpoints/multi-models-3.png":::
126
+
:::image type="content" source="media/concept-online-deployment-model-specification/multi-models-3.png" alt-text="A screenshot of a folder structure showing the model artifacts of a registered model." lightbox="media/concept-online-deployment-model-specification/multi-models-3.png":::
127
127
128
128
After you create your deployment, the environment variable `AZUREML_MODEL_DIR` points to the storage location within Azure where your models are stored. For example, `/var/azureml-app/azureml-models/local-multimodel/3` contains the models and the file structure. `AZUREML_MODEL_DIR` points to the folder containing the root of the model artifacts. Based on this example, the contents of the `AZUREML_MODEL_DIR` folder look like this:
129
129
130
-
:::image type="content" source="media/how-to-deploy-online-endpoints/multi-models-4.png" alt-text="A screenshot of the folder structure showing multiple models." lightbox="media/how-to-deploy-online-endpoints/multi-models-4.png":::
130
+
:::image type="content" source="media/concept-online-deployment-model-specification/multi-models-4.png" alt-text="A screenshot of the folder structure showing multiple models." lightbox="media/concept-online-deployment-model-specification/multi-models-4.png":::
131
131
132
132
Within your scoring script (`score.py`), you can load your models in the `init()` function. For example, load the `diabetes.sav` model:
> `Standard_DS1_v2` and `Standard_F2s_v2` may be too small for bigger models and may lead to container termination due to insufficient memory, not enough space on the disk, or probe failure as it takes too long to initiate the container. If you face [OutOfQuota errors](how-to-troubleshoot-online-endpoints.md?tabs=cli#error-outofquota) or [ReourceNotReady errors](how-to-troubleshoot-online-endpoints.md?tabs=cli#error-resourcenotready), try bigger VM SKUs. If you want to reduce the cost of deploying multiple models with managed online endpoint, see [the example for multi models](concept-endpoints-online.md#use-multiple-local-models-in-a-deployment).
33
+
> `Standard_DS1_v2` and `Standard_F2s_v2` may be too small for bigger models and may lead to container termination due to insufficient memory, not enough space on the disk, or probe failure as it takes too long to initiate the container. If you face [OutOfQuota errors](how-to-troubleshoot-online-endpoints.md?tabs=cli#error-outofquota) or [ReourceNotReady errors](how-to-troubleshoot-online-endpoints.md?tabs=cli#error-resourcenotready), try bigger VM SKUs. If you want to reduce the cost of deploying multiple models with managed online endpoint, see [Deployment configuration with several local models](concept-online-deployment-model-specification.md#deployment-configuration-with-several-local-models).
34
34
35
35
> [!NOTE]
36
36
> We recommend having more than 3 instances for deployments in production scenarios. In addition, Azure Machine Learning reserves 20% of your compute resources for performing upgrades on some VM SKUs as described in [Virtual machine quota allocation for deployment](how-to-manage-quotas.md#virtual-machine-quota-allocation-for-deployment). VM SKUs that are exempted from this extra quota reservation are listed below:
0 commit comments