Skip to content

Commit 3c16b10

Browse files
authored
Merge pull request #107128 from Blackmist/backing-out-changes
backing out changes as the feature referenced is still experimental
2 parents 15d9477 + 2a02c73 commit 3c16b10

File tree

2 files changed

+0
-83
lines changed

2 files changed

+0
-83
lines changed

articles/machine-learning/concept-model-management-and-deployment.md

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -66,11 +66,6 @@ Registered models are identified by name and version. Each time you register a m
6666
You can't delete a registered model that is being used in an active deployment.
6767
For more information, see the register model section of [Deploy models](how-to-deploy-and-where.md#registermodel).
6868

69-
### Profile models
70-
71-
Azure Machine Learning can help you understand the CPU and memory requirements of the service that will be created when you deploy your model. Profiling tests the service that runs your model and returns information such as the CPU usage, memory usage, and response latency. It also provides a CPU and memory recommendation based on the resource usage.
72-
For more information, see the profiling section of [Deploy models](how-to-deploy-and-where.md#profilemodel).
73-
7469
### Package and debug models
7570

7671
Before deploying a model into production, it is packaged into a Docker image. In most cases, image creation happens automatically in the background during deployment. You can manually specify the image.

articles/machine-learning/how-to-deploy-and-where.md

Lines changed: 0 additions & 78 deletions
Original file line numberDiff line numberDiff line change
@@ -179,8 +179,6 @@ To deploy the model as a service, you need the following components:
179179

180180
* **Inference configuration**. Inference configuration specifies the the environment configuration, entry script, and other components needed to run the model as a service.
181181

182-
Once you have the necessary components, you can profile the service that will be created as a result of deploying your model to understand its CPU and memory requirements.
183-
184182
### <a id="script"></a> 1. Define your entry script and dependencies
185183

186184
The entry script receives data submitted to a deployed web service and passes it to the model. It then takes the response returned by the model and returns that to the client. *The script is specific to your model*. It must understand the data that the model expects and returns.
@@ -518,82 +516,6 @@ In this example, the configuration specifies the following settings:
518516

519517
For information on using a custom Docker image with an inference configuration, see [How to deploy a model using a custom Docker image](how-to-deploy-custom-docker-image.md).
520518

521-
### <a id="profilemodel"></a> 3. Profile your model to determine resource utilization
522-
523-
Once you have registered your model and prepared the other components necessary for its deployment, you can determine the CPU and memory the deployed service will need. Profiling tests the service that runs your model and returns information such as the CPU usage, memory usage, and response latency. It also provides a recommendation for the CPU and memory based on resource usage.
524-
525-
In order to profile your model you will need:
526-
* A registered model.
527-
* An inference configuration based on your entry script and inference environment definition.
528-
* A single column tabular dataset, where each row contains a string representing sample request data.
529-
530-
> [!IMPORTANT]
531-
> At this point we only support profiling of services that expect their request data to be a string, for example: string serialized json, text, string serialized image, etc. The content of each row of the dataset (string) will be put into the body of the HTTP request and sent to the service encapsulating the model for scoring.
532-
533-
Below is an example of how you can construct an input dataset to profile a service which expects its incoming request data to contain serialized json. In this case we created a dataset based one hundred instances of the same request data content. In real world scenarios we suggest that you use larger datasets containing various inputs, especially if your model resource usage/behavior is input dependent.
534-
535-
```python
536-
import json
537-
from azureml.core import Datastore
538-
from azureml.core.dataset import Dataset
539-
from azureml.data import dataset_type_definitions
540-
541-
input_json = {'data': [[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
542-
[10, 9, 8, 7, 6, 5, 4, 3, 2, 1]]}
543-
# create a string that can be utf-8 encoded and
544-
# put in the body of the request
545-
serialized_input_json = json.dumps(input_json)
546-
dataset_content = []
547-
for i in range(100):
548-
dataset_content.append(serialized_input_json)
549-
dataset_content = '\n'.join(dataset_content)
550-
file_name = 'sample_request_data.txt'
551-
f = open(file_name, 'w')
552-
f.write(dataset_content)
553-
f.close()
554-
555-
# upload the txt file created above to the Datastore and create a dataset from it
556-
data_store = Datastore.get_default(ws)
557-
data_store.upload_files(['./' + file_name], target_path='sample_request_data')
558-
datastore_path = [(data_store, 'sample_request_data' +'/' + file_name)]
559-
sample_request_data = Dataset.Tabular.from_delimited_files(
560-
datastore_path, separator='\n',
561-
infer_column_types=True,
562-
header=dataset_type_definitions.PromoteHeadersBehavior.NO_HEADERS)
563-
sample_request_data = sample_request_data.register(workspace=ws,
564-
name='sample_request_data',
565-
create_new_version=True)
566-
```
567-
568-
Once you have the dataset containing sample request data ready, create an inference configuration. Inference configuration is based on the score.py and the environment definition. The following example demonstrates how to create the inference configuration and run profiling:
569-
570-
```python
571-
from azureml.core.model import InferenceConfig, Model
572-
from azureml.core.dataset import Dataset
573-
574-
575-
model = Model(ws, id=model_id)
576-
inference_config = InferenceConfig(entry_script='path-to-score.py',
577-
environment=myenv)
578-
input_dataset = Dataset.get_by_name(workspace=ws, name='sample_request_data')
579-
profile = Model.profile(ws,
580-
'unique_name',
581-
[model],
582-
inference_config,
583-
input_dataset=input_dataset)
584-
585-
profile.wait_for_completion(True)
586-
587-
# see the result
588-
details = profile.get_details()
589-
```
590-
591-
The following command demonstrates how to profile a model by using the CLI:
592-
593-
```azurecli-interactive
594-
az ml model profile -g <resource-group-name> -w <workspace-name> --inference-config-file <path-to-inf-config.json> -m <model-id> --idi <input-dataset-id> -n <unique-name>
595-
```
596-
597519
## Deploy to target
598520

599521
Deployment uses the inference configuration deployment configuration to deploy the models. The deployment process is similar regardless of the compute target. Deploying to AKS is slightly different because you must provide a reference to the AKS cluster.

0 commit comments

Comments
 (0)