Skip to content

Commit 894153e

Browse files
Merge pull request #113 from Blackmist/297771-fresh
freshness
2 parents df96310 + d228500 commit 894153e

File tree

2 files changed

+20
-19
lines changed

2 files changed

+20
-19
lines changed

articles/machine-learning/concept-environments.md

Lines changed: 19 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -5,18 +5,19 @@ description: Learn about machine learning environments, which enable reproducibl
55
services: machine-learning
66
ms.service: azure-machine-learning
77
ms.subservice: core
8-
ms.topic: conceptual
8+
ms.topic: concept-article
99
author: Blackmist
1010
ms.author: larryfr
1111
ms.reviewer: osiotugo
12-
ms.date: 01/03/2024
12+
ms.date: 09/04/2024
13+
ms.custom: FY25Q1-Linter
14+
# Customer Intent: As a data scientist, I want to understand what Azure Machine Learning environments are and how they can help me manage my machine learning dependencies.
1315
---
1416

1517
# What are Azure Machine Learning environments?
1618

17-
Azure Machine Learning environments are an encapsulation of the environment where your machine learning training happens. They specify the Python packages, and software settings around your training and scoring scripts. The environments are managed and versioned entities within your Machine Learning workspace that enable reproducible, auditable, and portable machine learning workflows across various compute targets.
19+
Azure Machine Learning environments are an encapsulation of the environment where your machine learning training or inferencing happens. They specify the Python packages, and software settings around your training and scoring scripts. The environments are managed and versioned entities within your Machine Learning workspace that enable reproducible, auditable, and portable machine learning workflows across various compute targets. You can use an `Environment` object to:
1820

19-
You can use an `Environment` object to:
2021
* Develop your training script.
2122
* Reuse the same environment on Azure Machine Learning Compute for model training at scale.
2223
* Deploy your model with that same environment.
@@ -32,15 +33,15 @@ The environment, compute target, and training script together form the job confi
3233

3334
Environments can broadly be divided into three categories: *curated*, *user-managed*, and *system-managed*.
3435

35-
Curated environments are provided by Azure Machine Learning and are available in your workspace by default. Intended to be used as is, they contain collections of Python packages and settings to help you get started with various machine learning frameworks. These precreated environments also allow for faster deployment time. Curated environments are hosted in [AzureML Registry](concept-machine-learning-registries-mlops.md). For a full list, see the [environments in azureml registry](https://ml.azure.com/registries/azureml/environments).
36+
Curated environments are provided by Azure Machine Learning and are available in your workspace by default. Intended to be used as is, they contain collections of Python packages and settings to help you get started with various machine learning frameworks. These precreated environments also allow for faster deployment time. Curated environments are hosted in the __AzureML registry__, which is a [machine learning registry](concept-machine-learning-registries-mlops.md) hosted by Microsoft. For a full list, see the [environments in AzureML registry](https://ml.azure.com/registries/azureml/environments).
3637

37-
In user-managed environments, you're responsible for setting up your environment and installing every package that your training script needs on the compute target. Also be sure to include any dependencies needed for model deployment. User managed environment can be BYOC (Bring Your Own Container) or Docker Build Context based that delegates image materialization to AzureML.
38+
In user-managed environments, you're responsible for setting up your environment and installing every package that your training script needs on the compute target. Also be sure to include any dependencies needed for model deployment. User managed environment can be BYOC (Bring Your Own Container) or Docker Build Context based that delegates image materialization to Azure Machine Learning. Similar to curated environments, you can share user-managed environments across workspaces by using a [machine learning registry](concept-machine-learning-registries-mlops.md) that you create and manage.
3839

3940
You use system-managed environments when you want [conda](https://conda.io/docs/) to manage the Python environment for you. A new conda environment is materialized from your conda specification on top of a base docker image.
4041

4142
## Create and manage environments
4243

43-
You can create environments from clients like the Azure Machine Learning Python SDK, Azure Machine Learning CLI, Environments page in Azure Machine Learning studio, and [VS Code extension](how-to-manage-resources-vscode.md#create-environment). Every client allows you to customize the base image, Dockerfile, and Python layer if needed.
44+
You can create environments from the Azure Machine Learning Python SDK, Azure Machine Learning CLI, Azure Machine Learning studio, and [VS Code extension](how-to-manage-resources-vscode.md#create-environment). Every client allows you to customize the base image, Dockerfile, and Python layer if needed.
4445

4546
For specific code samples, see the "Create an environment" section of [How to use environments](how-to-manage-environments-v2.md#create-a-custom-environment).
4647

@@ -52,50 +53,50 @@ Environments are also easily managed through your workspace, which allows you to
5253
* View changes to your environments over time, which ensures reproducibility.
5354
* Build Docker images automatically from your environments.
5455

55-
"Anonymous" environments are automatically registered in your workspace when you submit an experiment. They will not be listed but may be retrieved by version.
56+
"Anonymous" environments are automatically registered in your workspace when you submit an experiment. They aren't listed but you can use the version to retrieve them.
5657

5758
For code samples, see the "Manage environments" section of [How to use environments](how-to-manage-environments-v2.md#manage-environments).
5859

5960
## Environment building, caching, and reuse
6061

61-
Azure Machine Learning builds environment definitions into Docker images. It also caches the environments so they can be reused in subsequent training jobs and service endpoint deployments. Running a training script remotely requires the creation of a Docker image. By default, AzureML manages image build target on available workspace [serverless compute quota](how-to-use-serverless-compute.md) if no dedicated compute set for the workspace.
62+
Azure Machine Learning builds environment definitions into Docker images. It also caches the environments so they can be reused in subsequent training jobs and service endpoint deployments. Running a training script remotely requires the creation of a Docker image. By default, Azure Machine Learning manages image build target on available workspace [serverless compute quota](how-to-use-serverless-compute.md) if no dedicated compute set for the workspace.
6263

6364
> [!NOTE]
64-
> Any network restrictions in AzureML Workspace might require dedicated user managed image build compute setup. Please follow the steps to [secure workspace resources](how-to-secure-workspace-vnet.md).
65+
> Any network restrictions in Azure Machine Learning workspace might require dedicated user managed image build compute setup. Please follow the steps to [secure workspace resources](how-to-secure-workspace-vnet.md).
6566
6667
### Submitting a job using an environment
6768

68-
When you first submit a remote job using an environment or create environment instance manually, the Azure Machine Learning builds an image for the provided specification. Result image is cached in the container registry instance associated with the workspace. Curated environments are already cached in the AzureML Registry. At the start of the job execution, the image is retrieved by the compute target from the relevant container registry.
69+
When you first submit a remote job using an environment or create environment instance manually, the Azure Machine Learning builds an image for the provided specification. Result image is cached in the container registry instance associated with the workspace. Curated environments are already cached in the Azure Machine Learning Registry. At the start of the job execution, the compute target retrieves the image from the relevant container registry.
6970

7071
### Building environments as Docker images
7172

72-
If the image for a particular environment definition doesn't already exist in the container registry instance associated with AzureML Workspace, a new image is built. For system managed environments, the image build consists of two steps:
73+
If the image for a particular environment definition doesn't already exist in the container registry instance associated with Azure Machine Learning workspace, a new image is built. For system managed environments, the image build consists of two steps:
7374

7475
1. Downloading a base image, and executing any Docker steps
7576
2. Building a conda environment according to conda dependencies specified in the environment definition.
7677

77-
For user managed environments provided docker context will be build as is. In this case you're responsible for installing any Python packages, by including them in your base image, or specifying custom Docker steps.
78+
For user managed environments provided docker context builds as is. In this case you're responsible for installing any Python packages, by including them in your base image, or specifying custom Docker steps.
7879

7980
### Image caching and reuse
8081

8182
If you use the same environment definition for another job, Azure Machine Learning reuses the cached image from the container registry associated with your Workspace.
8283

8384
To view the details of a cached image, check the Environments page in Azure Machine Learning studio or use [`MLClient.environments`](/python/api/azure-ai-ml/azure.ai.ml.mlclient#azure-ai-ml-mlclient-environments) to get and inspect the environment.
8485

85-
To determine whether to reuse a cached image or build a new one, Azure Machine Learning computes a [hash value](https://en.wikipedia.org/wiki/Hash_table) from the environment definition and compares it to the hashes of existing environments. The hash serves as a unique identifier for an environment and is based on the environment definition's:
86+
To determine whether to reuse a cached image or build a new one, Azure Machine Learning computes a [hash value](https://en.wikipedia.org/wiki/Hash_table) from the environment definition. It then compares the hash to the hashes of existing environments. The hash serves as a unique identifier for an environment and is based on the environment definition's:
8687

8788
* Base image
8889
* Custom docker steps
8990
* Python packages
9091

91-
The hash isn't affected by the environment name or version. If you rename your environment or create a new one with the same settings and packages as another environment, then the hash value remains the same. However, environment definition changes like adding or removing a Python package or changing a package version changes the resulting hash value. Changing the order of dependencies or channels in an environment will also change the hash and require a new image build. Similarly, any change to a curated environment results in the creation of a custom environment.
92+
The environment name and version have no effect on the hash. If you rename your environment or create a new one with the same settings and packages as another environment, then the hash value remains the same. However, environment definition changes like adding or removing a Python package or changing a package version changes the resulting hash value. Changing the order of dependencies or channels in an environment changes the hash and requires a new image build. Similarly, any change to a curated environment results in the creation of a custom environment.
9293

9394
> [!NOTE]
9495
> You will not be able to submit any local changes to a curated environment without changing the name of the environment. The prefixes "AzureML-" and "Microsoft" are reserved exclusively for curated environments, and your job submission will fail if the name starts with either of them.
9596
96-
The environment's computed hash value is compared with those in the Workspace container registry. If there is a match, then the cached image is pulled and used, otherwise an image build is triggered.
97+
The environment's computed hash value is compared with the hashes in the workspace container registry. If there's a match, then the cached image is pulled and used, otherwise an image build is triggered.
9798

98-
The following diagram shows three environment definitions. Two of them have different names and versions but identical base images and Python packages, which results in the same hash and corresponding cached image. The third environment has different Python packages and versions, leading to a different hash and cached image.
99+
The following diagram shows three environment definitions. Two of them have different names and versions but identical base images and Python packages, which result in the same hash and corresponding cached image. The third environment has different Python packages and versions, leading to a different hash and cached image.
99100

100101
![Diagram of environment caching and Docker images](./media/concept-environments/environment-caching.png)
101102

@@ -123,7 +124,7 @@ For more information on the base images, see the following links:
123124
* [Use a custom container to deploy a model to an online endpoint](how-to-deploy-custom-container.md)
124125
* [Managing environments and container images](concept-vulnerability-management.md#managing-environments-and-container-images)
125126

126-
## Next steps
127+
## Related content
127128

128129
* Learn how to [create and use environments](how-to-use-environments.md) in Azure Machine Learning.
129130
* See the Python SDK reference documentation for the [environment class](/python/api/azure-ai-ml/azure.ai.ml.entities.environment).

articles/machine-learning/concept-machine-learning-registries-mlops.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ In the preceding scenarios, you might use different Azure Machine Learning works
2929

3030
## Cross-workspace MLOps with registries
3131

32-
A registry, much like a Git repository, decouples machine learning assets from workspaces and hosts the assets in a central location, making them available to all workspaces in your organization.
32+
A registry, much like a Git repository, decouples machine learning assets from workspaces and hosts the assets in a central location, making them available to all workspaces in your organization. You can use registries to store and share assets such as __models__, __environments__, __components__, and __datasets__.
3333

3434
To promote models across development, test, and production environments, you can start by iteratively developing a model in the development environment. When you have a good candidate model, you can publish it to a registry. You can then deploy the model from the registry to endpoints in different workspaces.
3535

0 commit comments

Comments
 (0)