Skip to content

Commit 5a2d8fa

Browse files
authored
Merge pull request #705 from GitHubber17/310675-c
Freshness: 310675 Azure Machine Learning
2 parents 3d61dc7 + 44190f4 commit 5a2d8fa

File tree

1 file changed

+38
-35
lines changed

1 file changed

+38
-35
lines changed

articles/machine-learning/v1/how-to-train-with-custom-image.md

Lines changed: 38 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,46 +1,51 @@
11
---
22
title: Train a model by using a custom Docker image
33
titleSuffix: Azure Machine Learning
4-
description: Learn how to use your own Docker images, or curated ones from Microsoft, to train models in Azure Machine Learning.
4+
description: Learn how to use your own Docker images, or curated images from Microsoft, to train models in Azure Machine Learning.
55
services: machine-learning
66
ms.service: azure-machine-learning
77
ms.subservice: core
88
ms.author: ssalgado
99
author: saachigopal
1010
ms.reviewer: ssalgado
11-
ms.date: 11/14/2023
11+
ms.date: 10/08/2024
1212
ms.topic: how-to
1313
ms.custom: UpdateFrequency5, sdkv1
14+
15+
#Customer intent: As a data scientist professional, I want to use Docker or other curated images in Azure Machine Learning, so I can train models.
1416
---
1517

1618
# Train a model by using a custom Docker image
1719

1820
[!INCLUDE [sdk v1](../includes/machine-learning-sdk-v1.md)]
1921

20-
In this article, learn how to use a custom Docker image when you're training models with Azure Machine Learning. You'll use the example scripts in this article to classify pet images by creating a convolutional neural network.
22+
This article describes how to use a custom Docker image to train models with Azure Machine Learning. Example scripts show how to classify images by creating a convolutional neural network.
2123

22-
Azure Machine Learning provides a default Docker base image. You can also use Azure Machine Learning environments to specify a different base image, such as one of the maintained [Azure Machine Learning base images](https://github.com/Azure/AzureML-Containers) or your own [custom image](../how-to-deploy-custom-container.md). Custom base images allow you to closely manage your dependencies and maintain tighter control over component versions when running training jobs.
24+
Azure Machine Learning provides a default Docker base image. You can also use Azure Machine Learning environments to specify a different base image, such as a maintained [Azure Machine Learning base image](https://github.com/Azure/AzureML-Containers) or your own [custom image](../how-to-deploy-custom-container.md). Custom base images allow you to closely manage your dependencies and maintain tighter control over component versions when you run training jobs.
2325

2426
## Prerequisites
2527

26-
Run the code on either of these environments:
28+
To run the example code, your configuration must include one of the following environments:
29+
30+
- Azure Machine Learning compute instance with a dedicated notebook server preloaded with the Machine Learning SDK and Samples repository.
31+
32+
This configuration requires no downloads or other installation. To prepare this environment, see [Create resources to get started](../quickstart-create-resources.md).
33+
34+
- Jupyter Notebook server. The following resources provide instructions to help you prepare this environment:
2735

28-
* Azure Machine Learning compute instance (no downloads or installation necessary):
29-
* Complete the [Create resources to get started](../quickstart-create-resources.md) tutorial to create a dedicated notebook server preloaded with the SDK and the sample repository.
30-
* Your own Jupyter Notebook server:
31-
* Create a [workspace configuration file](../how-to-configure-environment.md#local-and-dsvm-only-create-a-workspace-configuration-file).
32-
* Install the [Azure Machine Learning SDK](/python/api/overview/azure/ml/install).
33-
* Create an [Azure container registry](/azure/container-registry/) or other Docker registry that's available on the internet.
36+
- Create a [workspace configuration file](../how-to-configure-environment.md#local-and-dsvm-only-create-a-workspace-configuration-file).
37+
- Install the [Azure Machine Learning SDK](/python/api/overview/azure/ml/install).
38+
- Create an [Azure container registry](/azure/container-registry/) or other Docker registry available on the internet.
3439

3540
## Set up a training experiment
3641

37-
In this section, you set up your training experiment by initializing a workspace, defining your environment, and configuring a compute target.
42+
The first task is to set up your training experiment by initializing a Machine Learning workspace, defining your environment, and configuring a compute target.
3843

3944
### Initialize a workspace
4045

41-
The [Azure Machine Learning workspace](../concept-workspace.md) is the top-level resource for the service. It gives you a centralized place to work with all the artifacts that you create. In the Python SDK, you can access the workspace artifacts by creating a [`Workspace`](/python/api/azureml-core/azureml.core.workspace.workspace) object.
46+
The [Azure Machine Learning workspace](../concept-workspace.md) is the top-level resource for the service. It gives you a centralized place to work with all the artifacts you create. In the Python SDK, you can access the workspace artifacts by creating a [`Workspace`](/python/api/azureml-core/azureml.core.workspace.workspace) object.
4247

43-
Create a `Workspace` object from the config.json file that you created as a [prerequisite](#prerequisites).
48+
As needed, create a `Workspace` object from the config.json file that you created as a [prerequisite](#prerequisites).
4449

4550
```Python
4651
from azureml.core import Workspace
@@ -60,7 +65,7 @@ fastai_env = Environment("fastai2")
6065

6166
The specified base image in the following code supports the fast.ai library, which allows for distributed deep-learning capabilities. For more information, see the [fast.ai Docker Hub repository](https://hub.docker.com/u/fastdotai).
6267

63-
When you're using your custom Docker image, you might already have your Python environment properly set up. In that case, set the `user_managed_dependencies` flag to `True` to use your custom image's built-in Python environment. By default, Azure Machine Learning builds a Conda environment with dependencies that you specified. The service runs the script in that environment instead of using any Python libraries that you installed on the base image.
68+
When you use your custom Docker image, you might already have your Python environment properly set up. In that case, set the `user_managed_dependencies` flag to `True` to use your custom image's built-in Python environment. By default, Azure Machine Learning builds a Conda environment with dependencies that you specified. The service runs the script in that environment instead of using any Python libraries that you installed on the base image.
6469

6570
```python
6671
fastai_env.docker.base_image = "fastdotai/fastai2:latest"
@@ -72,7 +77,7 @@ fastai_env.python.user_managed_dependencies = True
7277
To use an image from a private container registry that isn't in your workspace, use `docker.base_image_registry` to specify the address of the repository and a username and password:
7378

7479
```python
75-
# Set the container registry information.
80+
# Set the container registry information
7681
fastai_env.docker.base_image_registry.address = "myregistry.azurecr.io"
7782
fastai_env.docker.base_image_registry.username = "username"
7883
fastai_env.docker.base_image_registry.password = "password"
@@ -83,27 +88,27 @@ fastai_env.docker.base_image_registry.password = "password"
8388
It's also possible to use a custom Dockerfile. Use this approach if you need to install non-Python packages as dependencies. Remember to set the base image to `None`.
8489

8590
```python
86-
# Specify Docker steps as a string.
91+
# Specify Docker steps as a string
8792
dockerfile = r"""
8893
FROM mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20210615.v1
8994
RUN echo "Hello from custom container!"
9095
"""
9196

92-
# Set the base image to None, because the image is defined by Dockerfile.
97+
# Set the base image to None, because the image is defined by Dockerfile
9398
fastai_env.docker.base_image = None
9499
fastai_env.docker.base_dockerfile = dockerfile
95100

96-
# Alternatively, load the string from a file.
101+
# Alternatively, load the string from a file
97102
fastai_env.docker.base_image = None
98103
fastai_env.docker.base_dockerfile = "./Dockerfile"
99104
```
100105

101-
>[!IMPORTANT]
106+
> [!IMPORTANT]
102107
> Azure Machine Learning only supports Docker images that provide the following software:
103-
> * Ubuntu 18.04 or greater.
104-
> * Conda 4.7.# or greater.
105-
> * Python 3.7+.
106-
> * A POSIX compliant shell available at /bin/sh is required in any container image used for training.
108+
> * Ubuntu 18.04 or greater
109+
> * Conda 4.7.# or greater
110+
> * Python 3.7+
111+
> * A POSIX compliant shell available at /bin/sh is required in any container image used for training
107112
108113
For more information about creating and managing Azure Machine Learning environments, see [Create and use software environments](../how-to-use-environments.md).
109114

@@ -119,7 +124,7 @@ As with other Azure services, there are limits on certain resources (for example
119124
from azureml.core.compute import ComputeTarget, AmlCompute
120125
from azureml.core.compute_target import ComputeTargetException
121126

122-
# Choose a name for your cluster.
127+
# Choose a name for your cluster
123128
cluster_name = "gpu-cluster"
124129

125130
try:
@@ -130,19 +135,17 @@ except ComputeTargetException:
130135
compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_NC6',
131136
max_nodes=4)
132137

133-
# Create the cluster.
138+
# Create the cluster
134139
compute_target = ComputeTarget.create(ws, cluster_name, compute_config)
135140

136141
compute_target.wait_for_completion(show_output=True)
137142

138-
# Use get_status() to get a detailed status for the current AmlCompute.
143+
# Use get_status() to get a detailed status for the current AmlCompute
139144
print(compute_target.get_status().serialize())
140145
```
141146

142-
143-
>[!IMPORTANT]
144-
>Use CPU SKUs for any image build on compute.
145-
147+
> [!IMPORTANT]
148+
> Use CPU SKUs for any image build on compute.
146149
147150
## Configure your training job
148151

@@ -173,7 +176,7 @@ run.wait_for_completion(show_output=True)
173176
> [!WARNING]
174177
> Azure Machine Learning runs training scripts by copying the entire source directory. If you have sensitive data that you don't want to upload, use an [.ignore file](../concept-train-machine-learning-model.md#understand-what-happens-when-you-submit-a-training-job) or don't include it in the source directory. Instead, access your data by using a [datastore](/python/api/azureml-core/azureml.data).
175178
176-
## Next steps
177-
In this article, you trained a model by using a custom Docker image. See these other articles to learn more about Azure Machine Learning:
178-
* [Track run metrics](../how-to-log-view-metrics.md) during training.
179-
* [Deploy a model](../how-to-deploy-custom-container.md) by using a custom Docker image.
179+
## Related content
180+
181+
* [Track run metrics](../how-to-log-view-metrics.md)
182+
* [Deploy a model by using a custom Docker image](../how-to-deploy-custom-container.md)

0 commit comments

Comments
 (0)