You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/concept-model-management-and-deployment.md
-5Lines changed: 0 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -66,11 +66,6 @@ Registered models are identified by name and version. Each time you register a m
66
66
You can't delete a registered model that is being used in an active deployment.
67
67
For more information, see the register model section of [Deploy models](how-to-deploy-and-where.md#registermodel).
68
68
69
-
### Profile models
70
-
71
-
Azure Machine Learning can help you understand the CPU and memory requirements of the service that will be created when you deploy your model. Profiling tests the service that runs your model and returns information such as the CPU usage, memory usage, and response latency. It also provides a CPU and memory recommendation based on the resource usage.
72
-
For more information, see the profiling section of [Deploy models](how-to-deploy-and-where.md#profilemodel).
73
-
74
69
### Package and debug models
75
70
76
71
Before deploying a model into production, it is packaged into a Docker image. In most cases, image creation happens automatically in the background during deployment. You can manually specify the image.
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-deploy-and-where.md
-78Lines changed: 0 additions & 78 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -179,8 +179,6 @@ To deploy the model as a service, you need the following components:
179
179
180
180
***Inference configuration**. Inference configuration specifies the the environment configuration, entry script, and other components needed to run the model as a service.
181
181
182
-
Once you have the necessary components, you can profile the service that will be created as a result of deploying your model to understand its CPUand memory requirements.
183
-
184
182
### <a id="script"></a> 1. Define your entry script and dependencies
185
183
186
184
The entry script receives data submitted to a deployed web service and passes it to the model. It then takes the response returned by the model and returns that to the client. *The script is specific to your model*. It must understand the data that the model expects and returns.
@@ -518,82 +516,6 @@ In this example, the configuration specifies the following settings:
518
516
519
517
For information on using a custom Docker image with an inference configuration, see [How to deploy a model using a custom Docker image](how-to-deploy-custom-docker-image.md).
520
518
521
-
### <a id="profilemodel"></a> 3. Profile your model to determine resource utilization
522
-
523
-
Once you have registered your model and prepared the other components necessary for its deployment, you can determine the CPUand memory the deployed service will need. Profiling tests the service that runs your model and returns information such as the CPU usage, memory usage, and response latency. It also provides a recommendation for the CPUand memory based on resource usage.
524
-
525
-
In order to profile your model you will need:
526
-
* A registered model.
527
-
* An inference configuration based on your entry script and inference environment definition.
528
-
* A single column tabular dataset, where each row contains a string representing sample request data.
529
-
530
-
> [!IMPORTANT]
531
-
> At this point we only support profiling of services that expect their request data to be a string, for example: string serialized json, text, string serialized image, etc. The content of each row of the dataset (string) will be put into the body of the HTTP request and sent to the service encapsulating the model for scoring.
532
-
533
-
Below is an example of how you can construct an input dataset to profile a service which expects its incoming request data to contain serialized json. In this case we created a dataset based one hundred instances of the same request data content. In real world scenarios we suggest that you use larger datasets containing various inputs, especially if your model resource usage/behavior isinput dependent.
Once you have the dataset containing sample request data ready, create an inference configuration. Inference configuration is based on the score.py and the environment definition. The following example demonstrates how to create the inference configuration and run profiling:
569
-
570
-
```python
571
-
from azureml.core.model import InferenceConfig, Model
The following command demonstrates how to profile a model by using the CLI:
592
-
593
-
```azurecli-interactive
594
-
az ml model profile -g <resource-group-name>-w <workspace-name>--inference-config-file<path-to-inf-config.json>-m <model-id>--idi <input-dataset-id>-n <unique-name>
595
-
```
596
-
597
519
## Deploy to target
598
520
599
521
Deployment uses the inference configuration deployment configuration to deploy the models. The deployment process is similar regardless of the compute target. Deploying to AKSis slightly different because you must provide a reference to the AKS cluster.
0 commit comments