You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/concept-environments.md
+8-16Lines changed: 8 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -55,11 +55,11 @@ For code samples, see the "Manage environments" section of [Reuse environments f
55
55
56
56
## Environment building, caching, and reuse
57
57
58
-
The environments within Azure Machine Learning service are managed by a microservice called Environment Management service. It is responsible for building environment definitions into Docker images and conda environments, and caching the environments so they can be reused in subsequent training runs and deployments.
58
+
The Azure Machine Learning service builds environment definitions into Docker images and conda environments. It also caches the environments so they can be reused in subsequent training runs and service endpoint deployments.
59
59
60
60
### Building environments as Docker images
61
61
62
-
Typically, when you first submit a run using an environment, the Environment Management service invokes an [ACR Build Task](https://docs.microsoft.com/azure/container-registry/container-registry-tasks-overview) on the Azure Container Registry (ACR) associated with the Workspace. The built Docker image is then cached on the Workspace ACR. At the start of the run execution, the image is retrieved by the compute target.
62
+
Typically, when you first submit a run using an environment, the Azure Machine Learning service invokes an [ACR Build Task](https://docs.microsoft.com/azure/container-registry/container-registry-tasks-overview) on the Azure Container Registry (ACR) associated with the Workspace. The built Docker image is then cached on the Workspace ACR. At the start of the run execution, the image is retrieved by the compute target.
63
63
64
64
The image build consists of two steps:
65
65
@@ -70,32 +70,24 @@ The second step is omitted if you specify [user-managed dependencies](https://do
70
70
71
71
### Image caching and reuse
72
72
73
-
If you use the same environment definition for another run, the Environment Management service reuses the cached image from the Workspace ACR.
73
+
If you use the same environment definition for another run, the Azure Machine Learning service reuses the cached image from the Workspace ACR.
74
74
75
75
To view the details of a cached image, use [Environment.get_image_details](https://docs.microsoft.com/python/api/azureml-core/azureml.core.environment.environment?view=azure-ml-py#get-image-details-workspace-) method.
76
76
77
-
To determine whether to reuse a cached image or build a new one, the Environment Management service computes [a hash value](https://en.wikipedia.org/wiki/Hash_table) from the environment definition and compares it to the hashes of existing environments. The hash is based on:
77
+
To determine whether to reuse a cached image or build a new one, the service computes [a hash value](https://en.wikipedia.org/wiki/Hash_table) from the environment definition and compares it to the hashes of existing environments. The hash is based on:
78
78
79
79
* Base image property value
80
80
* Custom docker steps property value
81
81
* List of Python packages in Conda definition
82
82
* List of packages in Spark definition
83
83
84
-
The hash doesn't depend on environment name or version. See the following diagram that shows three environment definitions. Two of them have different name and version, but identical base image and Python packages. They have the same hash and therefore correspond to the same cached image. The third environment has different Python packages and versions, and therefore corresponds to a different cached image.
84
+
The hash doesn't depend on environment name or version. Environment definition changes, such as adding or removing a Python package or changing the package version, causes the hash value to change and triggers an image rebuild. However, if you simply rename your environment or create a new environment with the exact properties and packages of an existing one, then the hash value remains the same and the cached image is used.
85
85
86
-

86
+
See the following diagram that shows three environment definitions. Two of them have different name and version, but identical base image and Python packages. They have the same hash and therefore correspond to the same cached image. The third environment has different Python packages and versions, and therefore corresponds to a different cached image.
87
87
88
-
For example, following changes on environment definition will change the hash value, and result in an image rebuild:
88
+

89
89
90
-
* Adding or removing a Python package
91
-
* Changing a pinned package version, for example ```numpy==0.15``` to ```numpy==0.16```.
92
-
93
-
Following operations won't change the hash value, and will result in a cached image being used:
94
-
95
-
* Renaming an environment
96
-
* Creating a new environment whose properties and Python package list exactly matches an existing environment.
97
-
98
-
If you create an environment with unpinned package dependency, for example ```numpy```, that environment will keep using the package version installed at the time of environment creation. Also, any future environment with matching definition will keep using the old version. To update the package, specify a version number to force image rebuild. Note that new dependencies, including nested ones will be installed that might break a previously working scenario
90
+
If you create an environment with unpinned package dependency, for example ```numpy```, that environment will keep using the package version installed at the time of environment creation. Also, any future environment with matching definition will keep using the old version. To update the package, specify a version number to force image rebuild, for example ```numpy==1.18.1```. Note that new dependencies, including nested ones will be installed that might break a previously working scenario
99
91
100
92
> [!WARNING]
101
93
> The [Environment.build](https://docs.microsoft.com/python/api/azureml-core/azureml.core.environment.environment?view=azure-ml-py#build-workspace-) method will rebuild the cached image, with possible side-effect of updating unpinned packages and breaking reproducibility for all environment definitions corresponding to that cached image.
0 commit comments