Skip to content

Commit 9a3d20f

Browse files
authored
Merge pull request #830 from MicrosoftDocs/main
10/15/2024 PM Publish
2 parents 119f18f + bdf31d8 commit 9a3d20f

File tree

8 files changed

+449
-167
lines changed

8 files changed

+449
-167
lines changed

articles/machine-learning/data-science-virtual-machine/dsvm-common-identity.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ ms.date: 04/10/2024
1414

1515
# Set up a common identity on a Data Science Virtual Machine
1616

17-
On a Microsoft Azure Virtual Machine (VM), or a Data Science Virtual Machine (DSVM), you create local user accounts while provisioning the VM. Users then authenticate to the VM with credentials for those user accounts. If you have multiple VMs that your users need to access, credential management can become difficult. To solve the problem, you can deploy common user accounts, and manage those accounts, through a standards-based identity provider. You can then use a single set of credentials to access multiple resources on Azure, including multiple DSVMs.
17+
On a Microsoft Azure Virtual Machine (VM), or a Data Science Virtual Machine (DSVM), you create local user accounts while provisioning the VM. Users then authenticate to the VM with credentials for those user accounts. If you have multiple VMs, and your users need to access them, credential management can become difficult. To solve the problem, you can deploy common user accounts, and manage those accounts, through a standards-based identity provider. You can then use a single set of credentials to access multiple resources on Azure, including multiple DSVMs.
1818

1919
Active Directory is a popular identity provider. Azure supports it both as a cloud service and as an on-premises directory. You can use Microsoft Entra ID or on-premises Active Directory to authenticate users on a standalone DSVM, or a cluster of DSVMs, in an Azure virtual machine scale set. To do this, you join the DSVM instances to an Active Directory domain.
2020

@@ -38,7 +38,7 @@ Microsoft Entra Domain Services makes it simple to manage your identities. It pr
3838

3939
The **User** pane opens, as shown in this screenshot:
4040

41-
:::image type="content" source="./media/add-user.png" alt-text="Screenshot showing the add user pane." lightbox="./media/add-user.png":::
41+
:::image type="content" source="./media/dsvm-common-identity/add-user.png" alt-text="Screenshot showing the add user pane." lightbox="./media/dsvm-common-identity/add-user.png":::
4242

4343
1. Enter information about the user, such as **Name** and **User name**. The domain name portion of the user name must be either the initial default domain name "[domain name].onmicrosoft.com" or a verified, non-federated [custom domain name](/azure/active-directory/fundamentals/add-custom-domain) such as "contoso.com."
4444

3.14 KB
Loading

articles/machine-learning/v1/how-to-train-with-custom-image.md

Lines changed: 38 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,46 +1,51 @@
11
---
22
title: Train a model by using a custom Docker image
33
titleSuffix: Azure Machine Learning
4-
description: Learn how to use your own Docker images, or curated ones from Microsoft, to train models in Azure Machine Learning.
4+
description: Learn how to use your own Docker images, or curated images from Microsoft, to train models in Azure Machine Learning.
55
services: machine-learning
66
ms.service: azure-machine-learning
77
ms.subservice: core
88
ms.author: ssalgado
99
author: saachigopal
1010
ms.reviewer: ssalgado
11-
ms.date: 11/14/2023
11+
ms.date: 10/08/2024
1212
ms.topic: how-to
1313
ms.custom: UpdateFrequency5, sdkv1
14+
15+
#Customer intent: As a data scientist professional, I want to use Docker or other curated images in Azure Machine Learning, so I can train models.
1416
---
1517

1618
# Train a model by using a custom Docker image
1719

1820
[!INCLUDE [sdk v1](../includes/machine-learning-sdk-v1.md)]
1921

20-
In this article, learn how to use a custom Docker image when you're training models with Azure Machine Learning. You'll use the example scripts in this article to classify pet images by creating a convolutional neural network.
22+
This article describes how to use a custom Docker image to train models with Azure Machine Learning. Example scripts show how to classify images by creating a convolutional neural network.
2123

22-
Azure Machine Learning provides a default Docker base image. You can also use Azure Machine Learning environments to specify a different base image, such as one of the maintained [Azure Machine Learning base images](https://github.com/Azure/AzureML-Containers) or your own [custom image](../how-to-deploy-custom-container.md). Custom base images allow you to closely manage your dependencies and maintain tighter control over component versions when running training jobs.
24+
Azure Machine Learning provides a default Docker base image. You can also use Azure Machine Learning environments to specify a different base image, such as a maintained [Azure Machine Learning base image](https://github.com/Azure/AzureML-Containers) or your own [custom image](../how-to-deploy-custom-container.md). Custom base images allow you to closely manage your dependencies and maintain tighter control over component versions when you run training jobs.
2325

2426
## Prerequisites
2527

26-
Run the code on either of these environments:
28+
To run the example code, your configuration must include one of the following environments:
29+
30+
- Azure Machine Learning compute instance with a dedicated notebook server preloaded with the Machine Learning SDK and Samples repository.
31+
32+
This configuration requires no downloads or other installation. To prepare this environment, see [Create resources to get started](../quickstart-create-resources.md).
33+
34+
- Jupyter Notebook server. The following resources provide instructions to help you prepare this environment:
2735

28-
* Azure Machine Learning compute instance (no downloads or installation necessary):
29-
* Complete the [Create resources to get started](../quickstart-create-resources.md) tutorial to create a dedicated notebook server preloaded with the SDK and the sample repository.
30-
* Your own Jupyter Notebook server:
31-
* Create a [workspace configuration file](../how-to-configure-environment.md#local-and-dsvm-only-create-a-workspace-configuration-file).
32-
* Install the [Azure Machine Learning SDK](/python/api/overview/azure/ml/install).
33-
* Create an [Azure container registry](/azure/container-registry/) or other Docker registry that's available on the internet.
36+
- Create a [workspace configuration file](../how-to-configure-environment.md#local-and-dsvm-only-create-a-workspace-configuration-file).
37+
- Install the [Azure Machine Learning SDK](/python/api/overview/azure/ml/install).
38+
- Create an [Azure container registry](/azure/container-registry/) or other Docker registry available on the internet.
3439

3540
## Set up a training experiment
3641

37-
In this section, you set up your training experiment by initializing a workspace, defining your environment, and configuring a compute target.
42+
The first task is to set up your training experiment by initializing a Machine Learning workspace, defining your environment, and configuring a compute target.
3843

3944
### Initialize a workspace
4045

41-
The [Azure Machine Learning workspace](../concept-workspace.md) is the top-level resource for the service. It gives you a centralized place to work with all the artifacts that you create. In the Python SDK, you can access the workspace artifacts by creating a [`Workspace`](/python/api/azureml-core/azureml.core.workspace.workspace) object.
46+
The [Azure Machine Learning workspace](../concept-workspace.md) is the top-level resource for the service. It gives you a centralized place to work with all the artifacts you create. In the Python SDK, you can access the workspace artifacts by creating a [`Workspace`](/python/api/azureml-core/azureml.core.workspace.workspace) object.
4247

43-
Create a `Workspace` object from the config.json file that you created as a [prerequisite](#prerequisites).
48+
As needed, create a `Workspace` object from the config.json file that you created as a [prerequisite](#prerequisites).
4449

4550
```Python
4651
from azureml.core import Workspace
@@ -60,7 +65,7 @@ fastai_env = Environment("fastai2")
6065

6166
The specified base image in the following code supports the fast.ai library, which allows for distributed deep-learning capabilities. For more information, see the [fast.ai Docker Hub repository](https://hub.docker.com/u/fastdotai).
6267

63-
When you're using your custom Docker image, you might already have your Python environment properly set up. In that case, set the `user_managed_dependencies` flag to `True` to use your custom image's built-in Python environment. By default, Azure Machine Learning builds a Conda environment with dependencies that you specified. The service runs the script in that environment instead of using any Python libraries that you installed on the base image.
68+
When you use your custom Docker image, you might already have your Python environment properly set up. In that case, set the `user_managed_dependencies` flag to `True` to use your custom image's built-in Python environment. By default, Azure Machine Learning builds a Conda environment with dependencies that you specified. The service runs the script in that environment instead of using any Python libraries that you installed on the base image.
6469

6570
```python
6671
fastai_env.docker.base_image = "fastdotai/fastai2:latest"
@@ -72,7 +77,7 @@ fastai_env.python.user_managed_dependencies = True
7277
To use an image from a private container registry that isn't in your workspace, use `docker.base_image_registry` to specify the address of the repository and a username and password:
7378

7479
```python
75-
# Set the container registry information.
80+
# Set the container registry information
7681
fastai_env.docker.base_image_registry.address = "myregistry.azurecr.io"
7782
fastai_env.docker.base_image_registry.username = "username"
7883
fastai_env.docker.base_image_registry.password = "password"
@@ -83,27 +88,27 @@ fastai_env.docker.base_image_registry.password = "password"
8388
It's also possible to use a custom Dockerfile. Use this approach if you need to install non-Python packages as dependencies. Remember to set the base image to `None`.
8489

8590
```python
86-
# Specify Docker steps as a string.
91+
# Specify Docker steps as a string
8792
dockerfile = r"""
8893
FROM mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20210615.v1
8994
RUN echo "Hello from custom container!"
9095
"""
9196

92-
# Set the base image to None, because the image is defined by Dockerfile.
97+
# Set the base image to None, because the image is defined by Dockerfile
9398
fastai_env.docker.base_image = None
9499
fastai_env.docker.base_dockerfile = dockerfile
95100

96-
# Alternatively, load the string from a file.
101+
# Alternatively, load the string from a file
97102
fastai_env.docker.base_image = None
98103
fastai_env.docker.base_dockerfile = "./Dockerfile"
99104
```
100105

101-
>[!IMPORTANT]
106+
> [!IMPORTANT]
102107
> Azure Machine Learning only supports Docker images that provide the following software:
103-
> * Ubuntu 18.04 or greater.
104-
> * Conda 4.7.# or greater.
105-
> * Python 3.7+.
106-
> * A POSIX compliant shell available at /bin/sh is required in any container image used for training.
108+
> * Ubuntu 18.04 or greater
109+
> * Conda 4.7.# or greater
110+
> * Python 3.7+
111+
> * A POSIX compliant shell available at /bin/sh is required in any container image used for training
107112
108113
For more information about creating and managing Azure Machine Learning environments, see [Create and use software environments](../how-to-use-environments.md).
109114

@@ -119,7 +124,7 @@ As with other Azure services, there are limits on certain resources (for example
119124
from azureml.core.compute import ComputeTarget, AmlCompute
120125
from azureml.core.compute_target import ComputeTargetException
121126

122-
# Choose a name for your cluster.
127+
# Choose a name for your cluster
123128
cluster_name = "gpu-cluster"
124129

125130
try:
@@ -130,19 +135,17 @@ except ComputeTargetException:
130135
compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_NC6',
131136
max_nodes=4)
132137

133-
# Create the cluster.
138+
# Create the cluster
134139
compute_target = ComputeTarget.create(ws, cluster_name, compute_config)
135140

136141
compute_target.wait_for_completion(show_output=True)
137142

138-
# Use get_status() to get a detailed status for the current AmlCompute.
143+
# Use get_status() to get a detailed status for the current AmlCompute
139144
print(compute_target.get_status().serialize())
140145
```
141146

142-
143-
>[!IMPORTANT]
144-
>Use CPU SKUs for any image build on compute.
145-
147+
> [!IMPORTANT]
148+
> Use CPU SKUs for any image build on compute.
146149
147150
## Configure your training job
148151

@@ -173,7 +176,7 @@ run.wait_for_completion(show_output=True)
173176
> [!WARNING]
174177
> Azure Machine Learning runs training scripts by copying the entire source directory. If you have sensitive data that you don't want to upload, use an [.ignore file](../concept-train-machine-learning-model.md#understand-what-happens-when-you-submit-a-training-job) or don't include it in the source directory. Instead, access your data by using a [datastore](/python/api/azureml-core/azureml.data).
175178
176-
## Next steps
177-
In this article, you trained a model by using a custom Docker image. See these other articles to learn more about Azure Machine Learning:
178-
* [Track run metrics](../how-to-log-view-metrics.md) during training.
179-
* [Deploy a model](../how-to-deploy-custom-container.md) by using a custom Docker image.
179+
## Related content
180+
181+
* [Track run metrics](../how-to-log-view-metrics.md)
182+
* [Deploy a model by using a custom Docker image](../how-to-deploy-custom-container.md)

articles/search/.openpublishing.redirection.search.json

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
{
22
"redirections": [
3+
{
4+
"source_path_from_root": "/articles/search/index-projections-concept-intro.md",
5+
"redirect_url": "/azure/search/search-how-to-define-index-projections",
6+
"redirect_document_id": true
7+
},
38
{
49
"source_path_from_root": "/articles/search/tutorial-javascript-overview.md",
510
"redirect_url": "/azure/search/tutorial-csharp-overview",

0 commit comments

Comments
 (0)