|
1 | 1 | ---
|
2 |
| -title: Specification of models to use in the deployment of an online endpoint |
| 2 | +title: Model specification for online deployments |
3 | 3 | titleSuffix: Azure Machine Learning
|
4 |
| -description: Learn to specify the model to use for deployments in Azure Machine Learning online endpoints. |
| 4 | +description: Specify the model to use in an Azure Machine Learning online endpoint's deployment. |
5 | 5 | services: machine-learning
|
6 | 6 | ms.service: machine-learning
|
7 | 7 | ms.subservice: inferencing
|
8 | 8 | author: dem108
|
9 | 9 | ms.author: sehan
|
10 | 10 | ms.reviewer: mopeakande
|
11 |
| -ms.date: 06/28/2024 |
| 11 | +ms.date: 08/09/2024 |
12 | 12 | reviewer: msakande
|
13 | 13 | ms.topic: concept-article
|
14 | 14 | ms.custom: how-to, devplatv2, cliv2, sdkv2
|
15 | 15 | ---
|
16 | 16 |
|
17 |
| -# Model specification in an online deployment configuration |
| 17 | +# Specify model to deploy for use in online endpoint |
18 | 18 |
|
19 | 19 | [!INCLUDE [dev v2](includes/machine-learning-dev-v2.md)]
|
20 | 20 |
|
21 |
| -In this article, you learn to use the deployment configuration to specify models that you want to use in online deployments. When deploying a model to an Azure Machine Learning online endpoint, you need to specify the model one of two ways: |
| 21 | +In this article, you learn about the different ways to specify models that you want to use in online deployments. When deploying a model to an Azure Machine Learning online endpoint, you need to specify the model in one of two ways: |
22 | 22 |
|
23 | 23 | - Provide the path to the model's location on your local computer
|
24 | 24 | - Provide a reference to a versioned model that is already registered in your workspace.
|
25 | 25 |
|
| 26 | +How you specify your model for an online endpoint's deployment depends on where the model is stored. |
26 | 27 | In Azure Machine Learning, after you create your deployment, the environment variable `AZUREML_MODEL_DIR` points to the storage location within Azure where your model is stored.
|
27 | 28 |
|
28 |
| -## Deployment configurations with models that are stored locally |
| 29 | +## Deployment for models that are stored locally |
29 | 30 |
|
30 |
| -The following local folder structure illustrates how you can specify models that are available locally on your machine for use in an online deployment. |
| 31 | +This section uses this example of a local folder structure to show how you can specify models for use in an online deployment: |
31 | 32 |
|
32 | 33 | :::image type="content" source="media/concept-online-deployment-model-specification/multi-models-1.png" alt-text="A screenshot showing a local folder structure containing multiple models." lightbox="media/concept-online-deployment-model-specification/multi-models-1.png":::
|
33 | 34 |
|
34 |
| -### Deployment configuration with a single local model |
| 35 | +### Deployment for a single local model |
35 | 36 |
|
36 |
| -To use a single model that you have on your local machine in a deployment, specify the `path` to the `model` in your deployment YAML. The following code is an example of the deployment YAML with the path `/Downloads/multi-models-sample/models/model_1/v1/sample_m1.pkl`: |
| 37 | +To use a single model that you have on your local machine in a deployment, specify the `path` to the `model` in your deployment YAML configuration file. The following code is an example of the deployment YAML with the local path `/Downloads/multi-models-sample/models/model_1/v1/sample_m1.pkl`: |
37 | 38 |
|
38 | 39 | ```yml
|
39 | 40 | $schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
|
@@ -61,11 +62,11 @@ def init():
|
61 | 62 | model = joblib.load(model_path)
|
62 | 63 | ```
|
63 | 64 |
|
64 |
| -### Deployment configuration with several local models |
| 65 | +### Deployment for several local models |
65 | 66 |
|
66 | 67 | Although the Azure CLI, Python SDK, and other client tools allow you to specify only one model per deployment in the deployment definition, you can still use multiple models in a deployment by registering a model folder that contains all the models as files or subdirectories. For more information on registering your assets, such as models, so that you can specify their registered names and versions during deployment, see [Register your model and environment](how-to-deploy-online-endpoints.md#register-your-model-and-environment).
|
67 | 68 |
|
68 |
| -In the previous example folder structure, you notice that there are several models in the `models` folder. To use these models, in your deployment YAML, you specify the path to the `models` folder as follows: |
| 69 | +In the example local folder structure, you notice that there are several models in the `models` folder. To use these models, in your deployment YAML, you specify the path to the `models` folder as follows: |
69 | 70 |
|
70 | 71 | ```yml
|
71 | 72 | $schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
|
@@ -102,9 +103,9 @@ For an example of how to deploy several models to one deployment, see [Deploy mu
|
102 | 103 | > [!TIP]
|
103 | 104 | > If you have more than 1500 files to register, consider compressing the files or subdirectories as .tar.gz when registering the models. To consume the models, you can unpack the files or subdirectories in the `init()` function of the scoring script. Alternatively, when you register the models, set the `azureml.unpack` property to `True`, to automatically unpack the files or subdirectories. In either case, unpacking the files happens once in the initialization stage.
|
104 | 105 |
|
105 |
| -## Deployment configuration with models that are registered in your workspace |
| 106 | +## Deployment for models that are registered in your workspace |
106 | 107 |
|
107 |
| -You can use registered models in your deployment specifications by referencing their names in your deployment YAML. For example, the following deployment YAML configuration specifies the registered `model` name as `azureml:local-multimodel:3`: |
| 108 | +You can use registered models in your deployment definition by referencing their names in your deployment YAML. For example, the following deployment YAML configuration specifies the registered `model` name as `azureml:local-multimodel:3`: |
108 | 109 |
|
109 | 110 | ```yml
|
110 | 111 | $schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
|
@@ -137,13 +138,14 @@ def init():
|
137 | 138 | model = joblib.load(model_path)
|
138 | 139 | ```
|
139 | 140 |
|
140 |
| -## Deployment configuration for models that are available in the model catalog |
| 141 | +## Deployment for models that are available in the model catalog |
141 | 142 |
|
142 |
| -For models in the model catalog, except for models under Azure OpenAI collection, you can use the model IDs that is shown in the model card to deploy them. Model IDs are in the format of `azureml://registries/{registry_name}/models/{model_name}/versions/{model_version}`. Some model cards include example notebooks that show how to use model ID for the deployment. |
| 143 | +For any model in the model catalog, except for models that are in the Azure OpenAI collection, you can use the model ID shown in the model's card for deployment. Model IDs are of the form `azureml://registries/{registry_name}/models/{model_name}/versions/{model_version}`. For example, a model ID for the _Meta Llama 3-8 B Instruct_ model is `azureml://registries/azureml-meta/models/Meta-Llama-3-8B-Instruct/versions/2`. |
| 144 | +Some model cards include example notebooks that show you how to use the model ID for the deployment. |
143 | 145 |
|
144 |
| -## Deployment configuration for models that are available in your organization's registry |
| 146 | +## Deployment for models that are available in your organization's registry |
145 | 147 |
|
146 |
| -Each model in the organization registries has model ID in the format of `azureml://registries/{registry_name}/models/{model_name}/versions/{model_version}`. Note that you may also use environments that are registered in the same registry. |
| 148 | +Each model in an organization's registry has a model ID of the form `azureml://registries/{registry_name}/models/{model_name}/versions/{model_version}`. You might also choose to use environments that are registered in the same registry. |
147 | 149 |
|
148 | 150 | ## Related content
|
149 | 151 |
|
|
0 commit comments