You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/v1/how-to-deploy-inferencing-gpus.md
+7-7Lines changed: 7 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ ms.subservice: inferencing
8
8
ms.author: larryfr
9
9
author: Blackmist
10
10
ms.reviewer: vaidyas
11
-
ms.date: 11/16/2022
11
+
ms.date: 03/07/2025
12
12
ms.topic: how-to
13
13
ms.custom: UpdateFrequency5, deploy, sdkv1
14
14
---
@@ -25,7 +25,7 @@ Inference, or model scoring, is the phase where the deployed model is used to ma
25
25
26
26
> [!IMPORTANT]
27
27
> When using the Azure Machine Learning __SDK v1__, GPU inference is only supported on Azure Kubernetes Service. When using the Azure Machine Learning __SDK v2__ or __CLI v2__, you can use an online endpoint for GPU inference. For more information, see [Deploy and score a machine learning model with an online endpoint](../how-to-deploy-online-endpoints.md).
28
-
28
+
>
29
29
> For inference using a __machine learning pipeline__, GPUs are only supported on Azure Machine Learning Compute. For more information on using ML pipelines, see [Tutorial: Build an Azure Machine Learning pipeline for batch scoring](../tutorial-pipeline-batch-scoring-classification.md).
30
30
31
31
> [!TIP]
@@ -93,7 +93,7 @@ except ComputeTargetException:
93
93
```
94
94
95
95
> [!IMPORTANT]
96
-
> Azure will bill you as long as the AKS cluster exists. Make sure to delete your AKS cluster when you're done with it.
96
+
> Azure bills you as long as the AKS cluster exists. Make sure to delete your AKS cluster when you're done with it.
97
97
98
98
For more information on using AKS with Azure Machine Learning, see [How to deploy to Azure Kubernetes Service](how-to-deploy-azure-kubernetes-service.md).
99
99
@@ -140,7 +140,7 @@ This file is named `score.py`. For more information on entry scripts, see [How a
140
140
141
141
## Define the conda environment
142
142
143
-
The conda environment file specifies the dependencies for the service. It includes dependencies required by both the model and the entry script. Please note that you must indicate azureml-defaults with verion >= 1.0.45 as a pip dependency, because it contains the functionality needed to host the model as a web service. The following YAML defines the environment for a TensorFlow model. It specifies `tensorflow-gpu`, which will make use of the GPU used in this deployment:
143
+
The conda environment file specifies the dependencies for the service. It includes dependencies required by both the model and the entry script. Note that you must indicate azureml-defaults with verion >= 1.0.45 as a pip dependency, because it contains the functionality needed to host the model as a web service. The following YAML defines the environment for a TensorFlow model. It specifies `tensorflow-gpu`, which will make use of the GPU used in this deployment:
144
144
145
145
```yaml
146
146
name: project_environment
@@ -163,7 +163,7 @@ For this example, the file is saved as `myenv.yml`.
163
163
## Define the deployment configuration
164
164
165
165
> [!IMPORTANT]
166
-
> AKS does not allow pods to share GPUs, you can have only as many replicas of a GPU-enabled web service as there are GPUs in the cluster.
166
+
> AKS doesn't allow pods to share GPUs. You can have only as many replicas of a GPU-enabled web service as there are GPUs in the cluster.
167
167
168
168
The deployment configuration defines the Azure Kubernetes Service environment used to run the web service:
169
169
@@ -180,7 +180,7 @@ For more information, see the reference documentation for [AksService.deploy_con
180
180
181
181
## Define the inference configuration
182
182
183
-
The inference configuration points to the entry script and an environment object, which uses a docker image with GPU support. Please note that the YAML file used for environment definition must list azureml-defaults with version >= 1.0.45 as a pip dependency, because it contains the functionality needed to host the model as a web service.
183
+
The inference configuration points to the entry script and an environment object, which uses a docker image with GPU support. Note that the YAML file used for environment definition must list azureml-defaults with version >= 1.0.45 as a pip dependency, because it contains the functionality needed to host the model as a web service.
184
184
185
185
```python
186
186
from azureml.core.model import InferenceConfig
@@ -281,7 +281,7 @@ For more information on creating a client application, see [Create client to con
281
281
If you created the AKS cluster specifically for this example, delete your resources after you're done.
282
282
283
283
> [!IMPORTANT]
284
-
> Azure bills you based on how long the AKS cluster is deployed. Make sure to clean it up after you are done with it.
284
+
> Azure bills you based on how long the AKS cluster is deployed. Make sure to clean it up after you're done with it.
0 commit comments