Skip to content

Commit c8adecd

Browse files
Court72Court72
authored
Merge pull request #185700 from saachigopal/patch-26
Update resource-curated-environments.md
2 parents 8a0b7ff + f05d802 commit c8adecd

File tree

1 file changed

+31
-154
lines changed

1 file changed

+31
-154
lines changed

‎articles/machine-learning/resource-curated-environments.md

Lines changed: 31 additions & 154 deletions
Original file line numberDiff line numberDiff line change
@@ -19,185 +19,62 @@ This article lists the curated environments with latest framework versions in Az
1919
> [!NOTE]
2020
> Use the [Python SDK](how-to-use-environments.md), [CLI](/cli/azure/ml/environment#az_ml_environment_list), or Azure Machine Learning [studio](how-to-manage-environments-in-studio.md) to get the full list of environments and their dependencies. For more information, see the [environments article](how-to-use-environments.md#use-a-curated-environment).
2121
22+
## Why should I use curated environments?
23+
24+
* Reduces training and deployment latency.
25+
* Improves training and deployment success rate.
26+
* Avoid unnecessary image builds.
27+
* Only have required dependencies and access right in the image/container. 
28+
29+
>[!IMPORTANT]
30+
> To view more information about curated environment packages and versions, visit the Environments tab in the Azure Machine Learning [studio](https://docs.microsoft.com/azure/machine-learning/how-to-manage-environments-in-studio).
31+
2232
## Training curated environments
2333

2434
### PyTorch
2535

26-
**Name**: AzureML-pytorch-1.10-ubuntu18.04-py38-cuda11-gpu
36+
**Name**: AzureML-pytorch-1.10-ubuntu18.04-py38-cuda11-gpu
2737
**Description**: An environment for deep learning with PyTorch containing the AzureML Python SDK and other python packages.
28-
29-
The following Dockerfile can be customized for your personal workflows.
30-
31-
```dockerfile
32-
FROM mcr.microsoft.com/azureml/openmpi4.1.0-cuda11.1-cudnn8-ubuntu18.04:20211221.v1
33-
34-
ENV AZUREML_CONDA_ENVIRONMENT_PATH /azureml-envs/pytorch-1.10
35-
36-
# Create conda environment
37-
RUN conda create -p $AZUREML_CONDA_ENVIRONMENT_PATH \
38-
python=3.8 \
39-
pip=20.2.4 \
40-
pytorch=1.10.0 \
41-
torchvision=0.11.1 \
42-
torchaudio=0.10.0 \
43-
cudatoolkit=11.1.1 \
44-
nvidia-apex=0.1.0 \
45-
gxx_linux-64 \
46-
-c anaconda -c pytorch -c conda-forge
47-
48-
# Prepend path to AzureML conda environment
49-
ENV PATH $AZUREML_CONDA_ENVIRONMENT_PATH/bin:$PATH
50-
51-
# Install pip dependencies
52-
RUN pip install 'matplotlib>=3.3,<3.4' \
53-
'psutil>=5.8,<5.9' \
54-
'tqdm>=4.59,<4.63' \
55-
'pandas>=1.3,<1.4' \
56-
'scipy>=1.5,<1.8' \
57-
'numpy>=1.10,<1.22' \
58-
'ipykernel~=6.0' \
59-
'azureml-core==1.37.0.post1' \
60-
'azureml-defaults==1.37.0' \
61-
'azureml-mlflow==1.37.0' \
62-
'azureml-telemetry==1.37.0' \
63-
'tensorboard==2.6.0' \
64-
'tensorflow-gpu==2.6.0' \
65-
'onnxruntime-gpu>=1.7,<1.10' \
66-
'horovod==0.23' \
67-
'future==0.18.2' \
68-
'torch-tb-profiler==0.3.1'
69-
70-
71-
# This is needed for mpi to locate libpython
72-
ENV LD_LIBRARY_PATH $AZUREML_CONDA_ENVIRONMENT_PATH/lib:$LD_LIBRARY_PATH
73-
```
38+
* GPU: Cuda11
39+
* OS: Ubuntu18.04
40+
* PyTorch: 1.10
7441

7542
Other available PyTorch environments:
7643
* AzureML-pytorch-1.9-ubuntu18.04-py37-cuda11-gpu
7744
* AzureML-pytorch-1.8-ubuntu18.04-py37-cuda11-gpu
7845
* AzureML-pytorch-1.7-ubuntu18.04-py37-cuda11-gpu
7946

47+
8048
### LightGBM
8149

8250
**Name**: AzureML-lightgbm-3.2-ubuntu18.04-py37-cpu
8351
**Description**: An environment for machine learning with Scikit-learn, LightGBM, XGBoost, Dask containing the AzureML Python SDK and other packages.
52+
* OS: Ubuntu18.04
53+
* Dask: 2021.6
54+
* LightGBM: 3.2
55+
* Scikit-learn: 0.24
56+
* XGBoost: 1.4
8457

85-
The following Dockerfile can be customized for your personal workflows.
86-
87-
```dockerfile
88-
FROM mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20211221.v1
89-
90-
ENV AZUREML_CONDA_ENVIRONMENT_PATH /azureml-envs/lightgbm
91-
92-
# Create conda environment
93-
RUN conda create -p $AZUREML_CONDA_ENVIRONMENT_PATH \
94-
python=3.7 pip=20.2.4
95-
96-
# Prepend path to AzureML conda environment
97-
ENV PATH $AZUREML_CONDA_ENVIRONMENT_PATH/bin:$PATH
98-
99-
# Install pip dependencies
100-
RUN HOROVOD_WITH_TENSORFLOW=1 \
101-
pip install 'matplotlib>=3.3,<3.4' \
102-
'psutil>=5.8,<5.9' \
103-
'tqdm>=4.59,<4.60' \
104-
'pandas>=1.1,<1.2' \
105-
'numpy>=1.10,<1.20' \
106-
'scipy~=1.5.0' \
107-
'scikit-learn~=0.24.1' \
108-
'xgboost~=1.4.0' \
109-
'lightgbm~=3.2.0' \
110-
'dask~=2021.6.0' \
111-
'distributed~=2021.6.0' \
112-
'dask-ml~=1.9.0' \
113-
'adlfs~=0.7.0' \
114-
'ipykernel~=6.0' \
115-
'azureml-core==1.37.0.post1' \
116-
'azureml-defaults==1.37.0' \
117-
'azureml-mlflow==1.37.0' \
118-
'azureml-telemetry==1.37.0'
119-
120-
# This is needed for mpi to locate libpython
121-
ENV LD_LIBRARY_PATH $AZUREML_CONDA_ENVIRONMENT_PATH/lib:$LD_LIBRARY_PATH
122-
```
12358

12459
### Sklearn
125-
**Name**: AzureML-sklearn-0.24-ubuntu18.04-py37-cuda11-gpu
60+
**Name**: AzureML-sklearn-1.0-ubuntu20.04-py38-cpu
12661
**Description**: An environment for tasks such as regression, clustering, and classification with Scikit-learn. Contains the AzureML Python SDK and other python packages.
62+
* OS: Ubuntu20.04
63+
* Scikit-learn: 1.0
12764

128-
The following Dockerfile can be customized for your personal workflows.
129-
130-
```dockerfile
131-
FROM mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20211221.v1
132-
133-
ENV AZUREML_CONDA_ENVIRONMENT_PATH /azureml-envs/sklearn-0.24.1
134-
135-
# Create conda environment
136-
RUN conda create -p $AZUREML_CONDA_ENVIRONMENT_PATH \
137-
python=3.7 pip=20.2.4
138-
139-
# Prepend path to AzureML conda environment
140-
ENV PATH $AZUREML_CONDA_ENVIRONMENT_PATH/bin:$PATH
141-
142-
# Install pip dependencies
143-
RUN pip install 'matplotlib>=3.3,<3.4' \
144-
'psutil>=5.8,<5.9' \
145-
'tqdm>=4.59,<4.60' \
146-
'pandas>=1.1,<1.2' \
147-
'scipy>=1.5,<1.6' \
148-
'numpy>=1.10,<1.20' \
149-
'ipykernel~=6.0' \
150-
'azureml-core==1.37.0.post1' \
151-
'azureml-defaults==1.37.0' \
152-
'azureml-mlflow==1.37.0' \
153-
'azureml-telemetry==1.37.0' \
154-
'scikit-learn==0.24.1'
65+
Other available Sklearn environments:
66+
* AzureML-sklearn-0.24-ubuntu18.04-py37-cpu
15567

156-
# This is needed for mpi to locate libpython
157-
ENV LD_LIBRARY_PATH $AZUREML_CONDA_ENVIRONMENT_PATH/lib:$LD_LIBRARY_PATH
158-
```
15968

16069
### TensorFlow
16170

16271
**Name**: AzureML-tensorflow-2.4-ubuntu18.04-py37-cuda11-gpu
16372
**Description**: An environment for deep learning with TensorFlow containing the AzureML Python SDK and other python packages.
73+
* GPU: Cuda11
74+
* Horovod: 2.4.1
75+
* OS: Ubuntu18.04
76+
* TensorFlow: 2.4
16477

165-
The following Dockerfile can be customized for your personal workflows.
166-
167-
```dockerfile
168-
FROM mcr.microsoft.com/azureml/openmpi4.1.0-cuda11.0.3-cudnn8-ubuntu18.04:20211221.v1
169-
170-
ENV AZUREML_CONDA_ENVIRONMENT_PATH /azureml-envs/tensorflow-2.4
171-
172-
# Create conda environment
173-
RUN conda create -p $AZUREML_CONDA_ENVIRONMENT_PATH \
174-
python=3.7 pip=20.2.4
175-
176-
# Prepend path to AzureML conda environment
177-
ENV PATH $AZUREML_CONDA_ENVIRONMENT_PATH/bin:$PATH
178-
179-
# Install pip dependencies
180-
RUN HOROVOD_WITH_TENSORFLOW=1 \
181-
pip install 'matplotlib>=3.3,<3.4' \
182-
'psutil>=5.8,<5.9' \
183-
'tqdm>=4.59,<4.60' \
184-
'pandas>=1.1,<1.2' \
185-
'scipy>=1.5,<1.6' \
186-
'numpy>=1.10,<1.20' \
187-
'ipykernel~=6.0' \
188-
'azureml-core==1.37.0.post1' \
189-
'azureml-defaults==1.37.0' \
190-
'azureml-mlflow==1.37.0' \
191-
'azureml-telemetry==1.37.0' \
192-
'tensorboard==2.4.0' \
193-
'tensorflow-gpu==2.4.1' \
194-
'tensorflow-datasets==4.3.0' \
195-
'onnxruntime-gpu>=1.7,<1.8' \
196-
'horovod[tensorflow-gpu]==0.21.3'
197-
198-
# This is needed for mpi to locate libpython
199-
ENV LD_LIBRARY_PATH $AZUREML_CONDA_ENVIRONMENT_PATH/lib:$LD_LIBRARY_PATH
200-
```
20178

20279
## Automated ML (AutoML)
20380

@@ -216,6 +93,6 @@ For more information on AutoML and Azure ML pipelines, see [use automated ML in
21693

21794
[!INCLUDE [list-of-inference-prebuilt-docker-images](../../includes/aml-inference-list-prebuilt-docker-images.md)]
21895

219-
## Security
220-
Version updates for supported environments are released every two weeks to address vulnerabilities no older than 30 days.
96+
## Support
97+
Version updates for supported environments, including the base images they reference, are released every two weeks to address vulnerabilities no older than 30 days. Based on usage, some environments may be deprecated (hidden from the product but usable) to support more common machine learning scenarios.
22198

0 commit comments

Comments
 (0)