You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-troubleshoot-environments.md
+45-6Lines changed: 45 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -53,12 +53,51 @@ Running a training script remotely requires the creation of a Docker image.
53
53
54
54
## Reproducibility and vulnerabilities
55
55
56
-
Over time vulnerabilities are discovered and Docker images that correspond to AzureML environments may be flagged by scanning tools.
57
-
Updates for AzureML based images are released regularly, with a commitment of no unpatched vulnerabilities older than 30 days in the latest version of the image.
58
-
It's your responsibility to evaluate the threat and address vulnerabilities in environments.
59
-
Not all the vulnerabilities are exploitable, so you need to use your judgment when choosing between reproducibility and resolving vulnerabilities.
60
-
> [!IMPORTANT]
61
-
> There's no guarantee that the same set of Python dependencies will be materialized with an image rebuild or for a new environment with the same set of Python dependencies.
56
+
### Vulnerabilities
57
+
58
+
Vulnerabilities can be addressed by upgrading to a newer version of a dependency or migrating to a different dependency that satisfies security
59
+
requirements. Mitigating vulnerabilities is time consuming and costly since it can require refractoring of code and infrastructure. With the prevalence
60
+
of open source software and the use of complicated nested dependencies, it's important to manage and keep track of vulnerabilities.
61
+
62
+
There are some ways to decrease the impact of vulnerabilities:
63
+
64
+
- Reduce your number of dependencies - use the minimal set of the dependencies for each scenario.
65
+
- Compartmentalize your environment so issues can be scoped and fixed in one place.
66
+
- Understand flagged vulnerabilities and their relevance to your scenario.
67
+
68
+
### Vulnerabilities vs Reproducibility
69
+
70
+
Reproducibility is one of the foundations of software development. While developing production code, a repeated operation must guarantee the same
71
+
result. Mitigating vulnerabilities can disrupt reproducibility by changing dependencies.
72
+
73
+
AzureML's primary focus is to guarantee reproducibility. Environments can broadly be divided into three categories: curated,
74
+
user-managed, and system-managed.
75
+
76
+
**Curated environments** are pre-created environments that are managed by Azure Machine Learning (AzureML) and are available by default in every AzureML workspace provisioned.
77
+
78
+
Intended to be used as is, they contain collections of Python packages and settings to help you get started with various machine learning frameworks.
79
+
These pre-created environments also allow for faster deployment time.
80
+
81
+
In **user-managed environments**, you're responsible for setting up your environment and installing every package that your training script needs on the
82
+
compute target and for model deployment. These types of environments are represented by two subtypes:
83
+
84
+
- BYOC (bring your own container): the user provides a Docker image to AzureML
85
+
- Docker build context: AzureML materializes the image from the user provided content
86
+
87
+
Once you install more dependencies on top of a Microsoft-provided image, or bring your own base image, vulnerability
88
+
management becomes your responsibility.
89
+
90
+
You use **system-managed environments** when you want conda to manage the Python environment for you. A new isolated conda environment is materialized
91
+
from your conda specification on top of a base Docker image. While Azure Machine Learning patches base images with each release, whether you use the
92
+
latest image may be a tradeoff between reproducibility and vulnerability management. So, it's your responsibility to choose the environment version used
93
+
for your jobs or model deployments while using system-managed environments.
94
+
95
+
Associated to your Azure Machine Learning workspace is an Azure Container Registry instance that's used as a cache for container images. Any image
96
+
materialized is pushed to the container registry and used if experimentation or deployment is triggered for the corresponding environment. Azure
97
+
Machine Learning does not delete any image from your container registry, and it's your responsibility to evaluate which images you need to maintain over time. Users
98
+
can monitor and maintain environment hygiene with [Microsoft Defender for Container Registry](../defender-for-cloud/defender-for-containers-vulnerability-assessment-azure.md)
99
+
to help scan images for vulnerabilities. To
100
+
automate this process based on triggers from Microsoft Defender, see [Automate responses to Microsoft Defender for Cloud triggers](../defender-for-cloud/workflow-automation.md).
0 commit comments