Skip to content

Commit 8a6fcb3

Browse files
msakandemsakande
committed
New article: specify-models-for online-deployment
1 parent b46025f commit 8a6fcb3

File tree

3 files changed

+146
-117
lines changed

3 files changed

+146
-117
lines changed

‎articles/machine-learning/concept-endpoints-online.md

Lines changed: 1 addition & 117 deletions
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,7 @@ The following table describes the key attributes of a deployment:
122122
|-----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
123123
| Name | The name of the deployment. |
124124
| Endpoint name | The name of the endpoint to create the deployment under. |
125-
| Model<sup>1</sup> | The model to use for the deployment. This value can be either a reference to an existing versioned model in the workspace or an inline model specification. For more information on how to track and specify the path to your model, see [Identify model path with respect to `AZUREML_MODEL_DIR`](#identify-model-path-with-respect-to-azureml_model_dir). |
125+
| Model<sup>1</sup> | The model to use for the deployment. This value can be either a reference to an existing versioned model in the workspace or an inline model specification. For more information on how to track and specify the path to your model, see [Model specification in an online deployment configuration](concept-online-deployment-model-specification.md). |
126126
| Code path | The path to the directory on the local development environment that contains all the Python source code for scoring the model. You can use nested directories and packages. |
127127
| Scoring script | The relative path to the scoring file in the source code directory. This Python code must have an `init()` function and a `run()` function. The `init()` function will be called after the model is created or updated (you can use it to cache the model in memory, for example). The `run()` function is called at every invocation of the endpoint to do the actual scoring and prediction. |
128128
| Environment<sup>1</sup> | The environment to host the model and code. This value can be either a reference to an existing versioned environment in the workspace or an inline environment specification. __Note:__ Microsoft regularly patches the base images for known security vulnerabilities. You'll need to redeploy your endpoint to use the patched image. If you provide your own image, you're responsible for updating it. For more information, see [Image patching](concept-environments.md#image-patching). |
@@ -136,122 +136,6 @@ The following table describes the key attributes of a deployment:
136136
137137
To learn how to deploy online endpoints using the CLI, SDK, studio, and ARM template, see [Deploy an ML model with an online endpoint](how-to-deploy-online-endpoints.md).
138138

139-
### Identify model path with respect to `AZUREML_MODEL_DIR`
140-
141-
When deploying your model to Azure Machine Learning, you need to specify the location of the model you wish to deploy as part of your deployment configuration. In Azure Machine Learning, the path to your model is tracked with the `AZUREML_MODEL_DIR` environment variable. By identifying the model path with respect to `AZUREML_MODEL_DIR`, you can deploy one or more models that are stored locally on your machine or deploy a model that is registered in your Azure Machine Learning workspace.
142-
143-
For illustration, we reference the following local folder structure for the first two cases where you deploy a single model or deploy multiple models that are stored locally:
144-
145-
:::image type="content" source="media/how-to-deploy-online-endpoints/multi-models-1.png" alt-text="A screenshot showing a folder structure containing multiple models." lightbox="media/how-to-deploy-online-endpoints/multi-models-1.png":::
146-
147-
#### Use a single local model in a deployment
148-
149-
To use a single model that you have on your local machine in a deployment, specify the `path` to the `model` in your deployment YAML. Here's an example of the deployment YAML with the path `/Downloads/multi-models-sample/models/model_1/v1/sample_m1.pkl`:
150-
151-
```yml
152-
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
153-
name: blue
154-
endpoint_name: my-endpoint
155-
model:
156-
  path: /Downloads/multi-models-sample/models/model_1/v1/sample_m1.pkl
157-
code_configuration:
158-
  code: ../../model-1/onlinescoring/
159-
  scoring_script: score.py
160-
environment:
161-
  conda_file: ../../model-1/environment/conda.yml
162-
  image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest
163-
instance_type: Standard_DS3_v2
164-
instance_count: 1
165-
```
166-
167-
After you create your deployment, the environment variable `AZUREML_MODEL_DIR` will point to the storage location within Azure where your model is stored. For example, `/var/azureml-app/azureml-models/81b3c48bbf62360c7edbbe9b280b9025/1` will contain the model `sample_m1.pkl`.
168-
169-
Within your scoring script (`score.py`), you can load your model (in this example, `sample_m1.pkl`) in the `init()` function:
170-
171-
```python
172-
def init():
173-
model_path = os.path.join(str(os.getenv("AZUREML_MODEL_DIR")), "sample_m1.pkl")
174-
model = joblib.load(model_path)
175-
```
176-
177-
#### Use multiple local models in a deployment
178-
179-
Although the Azure CLI, Python SDK, and other client tools allow you to specify only one model per deployment in the deployment definition, you can still use multiple models in a deployment by registering a model folder that contains all the models as files or subdirectories.
180-
181-
In the previous example folder structure, you notice that there are multiple models in the `models` folder. In your deployment YAML, you can specify the path to the `models` folder as follows:
182-
183-
```yml
184-
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
185-
name: blue
186-
endpoint_name: my-endpoint
187-
model:
188-
  path: /Downloads/multi-models-sample/models/
189-
code_configuration:
190-
  code: ../../model-1/onlinescoring/
191-
  scoring_script: score.py
192-
environment:
193-
  conda_file: ../../model-1/environment/conda.yml
194-
  image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest
195-
instance_type: Standard_DS3_v2
196-
instance_count: 1
197-
```
198-
199-
After you create your deployment, the environment variable `AZUREML_MODEL_DIR` will point to the storage location within Azure where your models are stored. For example, `/var/azureml-app/azureml-models/81b3c48bbf62360c7edbbe9b280b9025/1` will contain the models and the file structure.
200-
201-
For this example, the contents of the `AZUREML_MODEL_DIR` folder will look like this:
202-
203-
:::image type="content" source="media/how-to-deploy-online-endpoints/multi-models-2.png" alt-text="A screenshot of the folder structure of the storage location for multiple models." lightbox="media/how-to-deploy-online-endpoints/multi-models-2.png":::
204-
205-
Within your scoring script (`score.py`), you can load your models in the `init()` function. The following code loads the `sample_m1.pkl` model:
206-
207-
```python
208-
def init():
209-
model_path = os.path.join(str(os.getenv("AZUREML_MODEL_DIR")), "models","model_1","v1", "sample_m1.pkl ")
210-
model = joblib.load(model_path)
211-
```
212-
213-
For an example of how to deploy multiple models to one deployment, see [Deploy multiple models to one deployment (CLI example)](https://github.com/Azure/azureml-examples/blob/main/cli/endpoints/online/custom-container/minimal/multimodel) and [Deploy multiple models to one deployment (SDK example)](https://github.com/Azure/azureml-examples/blob/main/sdk/python/endpoints/online/custom-container/online-endpoints-custom-container-multimodel.ipynb).
214-
215-
> [!TIP]
216-
> If you have more than 1500 files to register, consider compressing the files or subdirectories as .tar.gz when registering the models. To consume the models, you can uncompress the files or subdirectories in the `init()` function from the scoring script. Alternatively, when you register the models, set the `azureml.unpack` property to `True`, to automatically uncompress the files or subdirectories. In either case, uncompression happens once in the initialization stage.
217-
218-
#### Use models registered in your Azure Machine Learning workspace in a deployment
219-
220-
To use one or more models, which are registered in your Azure Machine Learning workspace, in your deployment, specify the name of the registered model(s) in your deployment YAML. For example, the following deployment YAML configuration specifies the registered `model` name as `azureml:local-multimodel:3`:
221-
222-
```yml
223-
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
224-
name: blue
225-
endpoint_name: my-endpoint
226-
model: azureml:local-multimodel:3
227-
code_configuration:
228-
  code: ../../model-1/onlinescoring/
229-
  scoring_script: score.py
230-
environment:
231-
  conda_file: ../../model-1/environment/conda.yml
232-
  image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest
233-
instance_type: Standard_DS3_v2
234-
instance_count: 1
235-
```
236-
237-
For this example, consider that `local-multimodel:3` contains the following model artifacts, which can be viewed from the **Models** tab in the Azure Machine Learning studio:
238-
239-
:::image type="content" source="media/how-to-deploy-online-endpoints/multi-models-3.png" alt-text="A screenshot of the folder structure showing the model artifacts of the registered model." lightbox="media/how-to-deploy-online-endpoints/multi-models-3.png":::
240-
241-
After you create your deployment, the environment variable `AZUREML_MODEL_DIR` will point to the storage location within Azure where your models are stored. For example, `/var/azureml-app/azureml-models/local-multimodel/3` will contain the models and the file structure. `AZUREML_MODEL_DIR` will point to the folder containing the root of the model artifacts.
242-
Based on this example, the contents of the `AZUREML_MODEL_DIR` folder will look like this:
243-
244-
:::image type="content" source="media/how-to-deploy-online-endpoints/multi-models-4.png" alt-text="A screenshot of the folder structure showing multiple models." lightbox="media/how-to-deploy-online-endpoints/multi-models-4.png":::
245-
246-
Within your scoring script (`score.py`), you can load your models in the `init()` function. For example, load the `diabetes.sav` model:
247-
248-
```python
249-
def init():
250-
model_path = os.path.join(str(os.getenv("AZUREML_MODEL_DIR"), "models", "diabetes", "1", "diabetes.sav")
251-
model = joblib.load(model_path)
252-
```
253-
254-
255139
### Virtual machine quota allocation for deployment
256140

257141
[!INCLUDE [quota-allocation-online-deployment](includes/quota-allocation-online-deployment.md)]

0 commit comments

Comments
 (0)