You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
title: Create Azure Container for PyTorch Custom Curated Environment
3
+
titleSuffix: Azure Machine Learning
4
+
description: Create custom curated Azure Container for PyTorch Environments in Azure Machine Learning studio to run your machine learning models and reuse it in different scenarios.
5
+
services: machine-learning
6
+
author: sheetalarkadam
7
+
ms.author: parinitarahi
8
+
ms.reviewer: ssalgado
9
+
ms.service: machine-learning
10
+
ms.subservice: core
11
+
ms.topic: how-to
12
+
ms.date: 03/20/2023
13
+
---
14
+
15
+
# Create custom curated Azure Container for PyTorch Environments in Azure Machine Learning studio
16
+
17
+
If you're looking to extend curated environment and add Hugging Face (HF) transformers or datasets or any other external packages to be installed, Azure Machine Learning offers to create a new env with docker context containing ACPT curated environment as base image and additional packages on top of it as below.
18
+
19
+
## Prerequisites
20
+
21
+
Before following the steps in this article, make sure you have the following prerequisites:
22
+
23
+
- An Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the [free or paid version of Azure Machine Learning](https://azure.microsoft.com/en-us/free/).
24
+
25
+
- An Azure Machine Learning workspace. If you don't have one, use the steps in the [Quickstart: Create workspace resources](quickstart-create-resources.md) article to create one.
26
+
27
+
## Navigate to environments
28
+
29
+
In the [Azure Machine Learning studio](https://ml.azure.com/registries/environments), navigate to the "Environments" section by selecting the "Environments" option.
30
+
31
+
:::image type="content" source="./media/how-to-azure-container-for-pytorch-environment/navigate-to-environments.png" alt-text="Screenshot of navigating to environments from Azure Machine Learning studio." lightbox= "./media/how-to-azure-container-for-pytorch-environment/navigate-to-environments.png":::
32
+
33
+
## Navigate to curated environments
34
+
35
+
Navigate to curated environments and search "acpt" to list all the available ACPT curated environments. Selecting the environment shows details of the environment.
36
+
37
+
:::image type="content" source="./media/how-to-azure-container-for-pytorch-environment/navigate-to-curated-environments.png" alt-text="Screenshot of navigating to curated environments." lightbox= "./media/how-to-azure-container-for-pytorch-environment/navigate-to-curated-environments.png":::
38
+
39
+
40
+
## Get details of the curated environments
41
+
42
+
To create custom environment, you need the base docker image repository, which can be found in the "Description" section as "Azure Container Registry". Copy the "Azure Container Registry" name, which is used later when you create a new custom environment.
Go back and select the " Custom Environments" tab.
49
+
50
+
:::image type="content" source="./media/how-to-azure-container-for-pytorch-environment/navigate-to-custom-environment.png" alt-text="Screenshot of navigating to custom environments." lightbox= "./media/how-to-azure-container-for-pytorch-environment/navigate-to-custom-environment.png":::
51
+
52
+
## Create custom environments
53
+
54
+
Select **+ Create**. In the "Create Environment" window, name the environment, description and select "Create a new docker context" in Select environments type section.
55
+
56
+
:::image type="content" source="./media/how-to-azure-container-for-pytorch-environment/create-environment-window.png" alt-text="Screenshot of creating custom environment." lightbox= "./media/how-to-azure-container-for-pytorch-environment/create-environment-window.png":::
57
+
58
+
Paste the docker image name that you copied in previously. Configure your environment by declaring the base image and add any env variables you want to use and the packages that you want to include.
59
+
60
+
:::image type="content" source="./media/how-to-azure-container-for-pytorch-environment/configure-environment.png" alt-text="Screenshot of configuring the environment with name, packages with docker context." lightbox= "./media/how-to-azure-container-for-pytorch-environment/configure-environment.png":::
61
+
62
+
Review your environment settings, add any tags if needed and select on the **Create** button to create your custom environment.
63
+
64
+
That's it! You've now created a custom environment in Azure Machine Learning studio and can use it to run your machine learning models.
65
+
66
+
## Next steps
67
+
68
+
- Learn more about environment objects:
69
+
-[What are Azure Machine Learning Environments? ](concept-environments.md).
70
+
- Learn more about [curated environments](concept-environments.md).
71
+
- Learn more about [training models in AML](concept-train-machine-learning-model.md).
title: Create Azure Container for PyTorch Custom Curated Environment
3
+
titleSuffix: Azure Machine Learning
4
+
description: Create custom curated Azure Container for PyTorch Environments in Azure Machine Learning studio to run your machine learning models and reuse it in different scenarios.
5
+
services: machine-learning
6
+
author: sheetalarkadam
7
+
ms.author: parinitarahi
8
+
ms.reviewer: ssalgado
9
+
ms.service: machine-learning
10
+
ms.subservice: core
11
+
ms.topic: how-to
12
+
ms.date: 03/20/2023
13
+
---
14
+
15
+
# Create custom curated Azure Container for PyTorch Environments in Azure Machine Learning studio
16
+
17
+
If you're looking to extend curated environment and add Hugging Face (HF) transformers or datasets or any other external packages to be installed, Azure Machine Learning offers to create a new env with docker context containing ACPT curated environment as base image and additional packages on top of it as below.
18
+
19
+
## Prerequisites
20
+
21
+
Before following the steps in this article, make sure you have the following prerequisites:
22
+
23
+
- An Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the [free or paid version of Azure Machine Learning](https://azure.microsoft.com/en-us/free/).
24
+
25
+
- An Azure Machine Learning workspace. If you don't have one, use the steps in the [Quickstart: Create workspace resources](quickstart-create-resources.md) article to create one.
26
+
27
+
## Navigate to environments
28
+
29
+
In the [Azure Machine Learning studio](https://ml.azure.com/registries/environments), navigate to the "Environments" section by selecting the "Environments" option.
30
+
31
+
:::image type="content" source="./media/how-to-azure-container-for-pytorch-environment/navigate-to-environments.png" alt-text="Screenshot of navigating to environments from Azure Machine Learning studio." lightbox= "./media/how-to-azure-container-for-pytorch-environment/navigate-to-environments.png":::
32
+
33
+
## Navigate to curated environments
34
+
35
+
Navigate to curated environments and search "acpt" to list all the available ACPT curated environments. Selecting the environment shows details of the environment.
36
+
37
+
:::image type="content" source="./media/how-to-azure-container-for-pytorch-environment/navigate-to-curated-environments.png" alt-text="Screenshot of navigating to curated environments." lightbox= "./media/how-to-azure-container-for-pytorch-environment/navigate-to-curated-environments.png":::
38
+
39
+
40
+
## Get details of the curated environments
41
+
42
+
To create custom environment, you need the base docker image repository, which can be found in the "Description" section as "Azure Container Registry". Copy the "Azure Container Registry" name, which is used later when you create a new custom environment.
Go back and select the " Custom Environments" tab.
49
+
50
+
:::image type="content" source="./media/how-to-azure-container-for-pytorch-environment/navigate-to-custom-environment.png" alt-text="Screenshot of navigating to custom environments." lightbox= "./media/how-to-azure-container-for-pytorch-environment/navigate-to-custom-environment.png":::
51
+
52
+
## Create custom environments
53
+
54
+
Select **+ Create**. In the "Create Environment" window, name the environment, description and select "Create a new docker context" in Select environments type section.
55
+
56
+
:::image type="content" source="./media/how-to-azure-container-for-pytorch-environment/create-environment-window.png" alt-text="Screenshot of creating custom environment." lightbox= "./media/how-to-azure-container-for-pytorch-environment/create-environment-window.png":::
57
+
58
+
Paste the docker image name that you copied in previously. Configure your environment by declaring the base image and add any env variables you want to use and the packages that you want to include.
59
+
60
+
:::image type="content" source="./media/how-to-azure-container-for-pytorch-environment/configure-environment.png" alt-text="Screenshot of configuring the environment with name, packages with docker context." lightbox= "./media/how-to-azure-container-for-pytorch-environment/configure-environment.png":::
61
+
62
+
Review your environment settings, add any tags if needed and select on the **Create** button to create your custom environment.
63
+
64
+
That's it! You've now created a custom environment in Azure Machine Learning studio and can use it to run your machine learning models.
65
+
66
+
## Next steps
67
+
68
+
- Learn more about environment objects:
69
+
-[What are Azure Machine Learning Environments? ](concept-environments.md).
70
+
- Learn more about [curated environments](concept-environments.md).
71
+
- Learn more about [training models in AML](concept-train-machine-learning-model.md).
Copy file name to clipboardExpand all lines: articles/machine-learning/resource-azure-container-for-pytorch.md
+12-9Lines changed: 12 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,7 @@ ms.reviewer: ssalgado
9
9
ms.service: machine-learning
10
10
ms.subservice: core
11
11
ms.topic: reference
12
-
ms.date: 03/17/2023
12
+
ms.date: 03/20/2023
13
13
---
14
14
15
15
# Azure Container for PyTorch (ACPT)
@@ -19,14 +19,13 @@ Azure Container for PyTorch is a lightweight, standalone environment that includ
19
19
> [!NOTE]
20
20
> Use the [Python SDK](how-to-use-environments.md), [CLI](/cli/azure/ml/environment#az-ml-environment-list), or Azure Machine Learning [studio](how-to-manage-environments-in-studio.md) to get the full list of environments and their dependencies. For more information, see the [environments article](how-to-use-environments.md#use-a-curated-environment).
21
21
22
-
23
22
## Why should I use ACPT?
24
23
25
24
* Use as is with preinstalled packages or build on top of the curated environment.
26
25
* Optimized training framework to set up, develop, accelerate PyTorch model on large workloads.
27
26
* Up-to-date stack with the latest compatible versions of Ubuntu, Python, PyTorch, CUDA\RocM, etc.
28
27
* Ease of use: All components installed and validated against dozens of Microsoft workloads to reduce setup costs and accelerate time to value.
29
-
* Latest Training Optimization Technologies: [ONNX RunTime](https://onnxruntime.ai/) , [DeepSpeed](https://www.deepspeed.ai/),[MSCCL](https://github.com/microsoft/msccl),and others.
28
+
* Latest Training Optimization Technologies: [ONNX RunTime](https://onnxruntime.ai/) , [DeepSpeed](https://www.deepspeed.ai/),[MSCCL](https://github.com/microsoft/msccl),and others.
30
29
* Integration with Azure Machine Learning: Track your PyTorch experiments on Azure Machine Learning studio or using the SDK.
31
30
* The image is also available as a [Data Science Virtual Machine (DSVM)](https://azure.microsoft.com/products/virtual-machines/data-science-virtual-machines/). To learn more about Data Science Virtual Machines, see [the DSVM overview documentation](data-science-virtual-machine/overview.md).
32
31
* Azure customer support reduces training and deployment latency.
@@ -45,15 +44,19 @@ The following configurations are supported:
45
44
46
45
| Environment Name | OS | GPU Version| Python Version | PyTorch Version | ORT-training Version | DeepSpeed Version | torch-ort Version |
Other packages like fairscale, horovod, msccl, protobuf, pyspark, pytest, pytorch-lightning, tensorboard, NebulaML, torchvision, torchmetrics to support all training needs
56
56
57
+
[!NOTE]
58
+
> Currently, due to underlying cuda and cluster incompatibilities, on [NC series](../virtual-machines/nc-series.md) only acpt-pytorch-1.11-cuda11.3 with cuda 11.3 and torch 1.11 can be used.
59
+
57
60
## Support
58
61
59
62
Version updates for supported environments, including the base images they reference, are released every two weeks to address vulnerabilities no older than 30 days. Based on usage, some environments may be deprecated (hidden from the product but usable) to support more common machine learning scenarios.
0 commit comments