Skip to content

Commit be24198

Browse files
authored
Update resource-curated-environments.md
1 parent 935aa80 commit be24198

File tree

1 file changed

+30
-153
lines changed

1 file changed

+30
-153
lines changed

articles/machine-learning/resource-curated-environments.md

Lines changed: 30 additions & 153 deletions
Original file line numberDiff line numberDiff line change
@@ -19,185 +19,62 @@ This article lists the curated environments with latest framework versions in Az
1919
> [!NOTE]
2020
> Use the [Python SDK](how-to-use-environments.md), [CLI](/cli/azure/ml/environment#az_ml_environment_list), or Azure Machine Learning [studio](how-to-manage-environments-in-studio.md) to get the full list of environments and their dependencies. For more information, see the [environments article](how-to-use-environments.md#use-a-curated-environment).
2121
22+
## Why should I use curated environments?
23+
24+
* Reduces training and deployment latency.
25+
* Improves training and deployment success rate.
26+
* Avoid unnecessary image builds.
27+
* Only have required dependencies and access right in the image/container. 
28+
29+
>[!IMPORTANT]
30+
> To see view more information about curated environment packages and versions, visit the Environments tab in the Azure Machine Learning studio.
31+
2232
## Training curated environments
2333

2434
### PyTorch
2535

26-
**Name**: AzureML-pytorch-1.10-ubuntu18.04-py38-cuda11-gpu
36+
**Name**: AzureML-pytorch-1.10-ubuntu18.04-py38-cuda11-gpu
2737
**Description**: An environment for deep learning with PyTorch containing the AzureML Python SDK and other python packages.
28-
29-
The following Dockerfile can be customized for your personal workflows.
30-
31-
```dockerfile
32-
FROM mcr.microsoft.com/azureml/openmpi4.1.0-cuda11.1-cudnn8-ubuntu18.04:20211221.v1
33-
34-
ENV AZUREML_CONDA_ENVIRONMENT_PATH /azureml-envs/pytorch-1.10
35-
36-
# Create conda environment
37-
RUN conda create -p $AZUREML_CONDA_ENVIRONMENT_PATH \
38-
python=3.8 \
39-
pip=20.2.4 \
40-
pytorch=1.10.0 \
41-
torchvision=0.11.1 \
42-
torchaudio=0.10.0 \
43-
cudatoolkit=11.1.1 \
44-
nvidia-apex=0.1.0 \
45-
gxx_linux-64 \
46-
-c anaconda -c pytorch -c conda-forge
47-
48-
# Prepend path to AzureML conda environment
49-
ENV PATH $AZUREML_CONDA_ENVIRONMENT_PATH/bin:$PATH
50-
51-
# Install pip dependencies
52-
RUN pip install 'matplotlib>=3.3,<3.4' \
53-
'psutil>=5.8,<5.9' \
54-
'tqdm>=4.59,<4.63' \
55-
'pandas>=1.3,<1.4' \
56-
'scipy>=1.5,<1.8' \
57-
'numpy>=1.10,<1.22' \
58-
'ipykernel~=6.0' \
59-
'azureml-core==1.37.0.post1' \
60-
'azureml-defaults==1.37.0' \
61-
'azureml-mlflow==1.37.0' \
62-
'azureml-telemetry==1.37.0' \
63-
'tensorboard==2.6.0' \
64-
'tensorflow-gpu==2.6.0' \
65-
'onnxruntime-gpu>=1.7,<1.10' \
66-
'horovod==0.23' \
67-
'future==0.18.2' \
68-
'torch-tb-profiler==0.3.1'
69-
70-
71-
# This is needed for mpi to locate libpython
72-
ENV LD_LIBRARY_PATH $AZUREML_CONDA_ENVIRONMENT_PATH/lib:$LD_LIBRARY_PATH
73-
```
38+
* GPU: Cuda11
39+
* OS: Ubuntu18.04
40+
* PyTorch: 1.10
7441

7542
Other available PyTorch environments:
7643
* AzureML-pytorch-1.9-ubuntu18.04-py37-cuda11-gpu
7744
* AzureML-pytorch-1.8-ubuntu18.04-py37-cuda11-gpu
7845
* AzureML-pytorch-1.7-ubuntu18.04-py37-cuda11-gpu
7946

47+
8048
### LightGBM
8149

8250
**Name**: AzureML-lightgbm-3.2-ubuntu18.04-py37-cpu
8351
**Description**: An environment for machine learning with Scikit-learn, LightGBM, XGBoost, Dask containing the AzureML Python SDK and other packages.
52+
* OS: Ubuntu18.04
53+
* Dask: 2021.6
54+
* LightGBM: 3.2
55+
* Scikit-learn: 0.24
56+
* XGBoost: 1.4
8457

85-
The following Dockerfile can be customized for your personal workflows.
86-
87-
```dockerfile
88-
FROM mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20211221.v1
89-
90-
ENV AZUREML_CONDA_ENVIRONMENT_PATH /azureml-envs/lightgbm
91-
92-
# Create conda environment
93-
RUN conda create -p $AZUREML_CONDA_ENVIRONMENT_PATH \
94-
python=3.7 pip=20.2.4
95-
96-
# Prepend path to AzureML conda environment
97-
ENV PATH $AZUREML_CONDA_ENVIRONMENT_PATH/bin:$PATH
98-
99-
# Install pip dependencies
100-
RUN HOROVOD_WITH_TENSORFLOW=1 \
101-
pip install 'matplotlib>=3.3,<3.4' \
102-
'psutil>=5.8,<5.9' \
103-
'tqdm>=4.59,<4.60' \
104-
'pandas>=1.1,<1.2' \
105-
'numpy>=1.10,<1.20' \
106-
'scipy~=1.5.0' \
107-
'scikit-learn~=0.24.1' \
108-
'xgboost~=1.4.0' \
109-
'lightgbm~=3.2.0' \
110-
'dask~=2021.6.0' \
111-
'distributed~=2021.6.0' \
112-
'dask-ml~=1.9.0' \
113-
'adlfs~=0.7.0' \
114-
'ipykernel~=6.0' \
115-
'azureml-core==1.37.0.post1' \
116-
'azureml-defaults==1.37.0' \
117-
'azureml-mlflow==1.37.0' \
118-
'azureml-telemetry==1.37.0'
119-
120-
# This is needed for mpi to locate libpython
121-
ENV LD_LIBRARY_PATH $AZUREML_CONDA_ENVIRONMENT_PATH/lib:$LD_LIBRARY_PATH
122-
```
12358

12459
### Sklearn
125-
**Name**: AzureML-sklearn-0.24-ubuntu18.04-py37-cuda11-gpu
60+
**Name**: AzureML-sklearn-1.0-ubuntu20.04-py38-cpu
12661
**Description**: An environment for tasks such as regression, clustering, and classification with Scikit-learn. Contains the AzureML Python SDK and other python packages.
62+
* OS: Ubuntu20.04
63+
* Scikit-learn: 1.0
12764

128-
The following Dockerfile can be customized for your personal workflows.
129-
130-
```dockerfile
131-
FROM mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20211221.v1
132-
133-
ENV AZUREML_CONDA_ENVIRONMENT_PATH /azureml-envs/sklearn-0.24.1
134-
135-
# Create conda environment
136-
RUN conda create -p $AZUREML_CONDA_ENVIRONMENT_PATH \
137-
python=3.7 pip=20.2.4
138-
139-
# Prepend path to AzureML conda environment
140-
ENV PATH $AZUREML_CONDA_ENVIRONMENT_PATH/bin:$PATH
141-
142-
# Install pip dependencies
143-
RUN pip install 'matplotlib>=3.3,<3.4' \
144-
'psutil>=5.8,<5.9' \
145-
'tqdm>=4.59,<4.60' \
146-
'pandas>=1.1,<1.2' \
147-
'scipy>=1.5,<1.6' \
148-
'numpy>=1.10,<1.20' \
149-
'ipykernel~=6.0' \
150-
'azureml-core==1.37.0.post1' \
151-
'azureml-defaults==1.37.0' \
152-
'azureml-mlflow==1.37.0' \
153-
'azureml-telemetry==1.37.0' \
154-
'scikit-learn==0.24.1'
65+
Other available Sklearn environments:
66+
* AzureML-sklearn-0.24-ubuntu18.04-py37-cpu
15567

156-
# This is needed for mpi to locate libpython
157-
ENV LD_LIBRARY_PATH $AZUREML_CONDA_ENVIRONMENT_PATH/lib:$LD_LIBRARY_PATH
158-
```
15968

16069
### TensorFlow
16170

16271
**Name**: AzureML-tensorflow-2.4-ubuntu18.04-py37-cuda11-gpu
16372
**Description**: An environment for deep learning with TensorFlow containing the AzureML Python SDK and other python packages.
73+
* GPU: Cuda11
74+
* Horovod: 2.4.1
75+
* OS: Ubuntu18.04
76+
* TensorFlow: 2.4
16477

165-
The following Dockerfile can be customized for your personal workflows.
166-
167-
```dockerfile
168-
FROM mcr.microsoft.com/azureml/openmpi4.1.0-cuda11.0.3-cudnn8-ubuntu18.04:20211221.v1
169-
170-
ENV AZUREML_CONDA_ENVIRONMENT_PATH /azureml-envs/tensorflow-2.4
171-
172-
# Create conda environment
173-
RUN conda create -p $AZUREML_CONDA_ENVIRONMENT_PATH \
174-
python=3.7 pip=20.2.4
175-
176-
# Prepend path to AzureML conda environment
177-
ENV PATH $AZUREML_CONDA_ENVIRONMENT_PATH/bin:$PATH
178-
179-
# Install pip dependencies
180-
RUN HOROVOD_WITH_TENSORFLOW=1 \
181-
pip install 'matplotlib>=3.3,<3.4' \
182-
'psutil>=5.8,<5.9' \
183-
'tqdm>=4.59,<4.60' \
184-
'pandas>=1.1,<1.2' \
185-
'scipy>=1.5,<1.6' \
186-
'numpy>=1.10,<1.20' \
187-
'ipykernel~=6.0' \
188-
'azureml-core==1.37.0.post1' \
189-
'azureml-defaults==1.37.0' \
190-
'azureml-mlflow==1.37.0' \
191-
'azureml-telemetry==1.37.0' \
192-
'tensorboard==2.4.0' \
193-
'tensorflow-gpu==2.4.1' \
194-
'tensorflow-datasets==4.3.0' \
195-
'onnxruntime-gpu>=1.7,<1.8' \
196-
'horovod[tensorflow-gpu]==0.21.3'
197-
198-
# This is needed for mpi to locate libpython
199-
ENV LD_LIBRARY_PATH $AZUREML_CONDA_ENVIRONMENT_PATH/lib:$LD_LIBRARY_PATH
200-
```
20178

20279
## Automated ML (AutoML)
20380

@@ -216,6 +93,6 @@ For more information on AutoML and Azure ML pipelines, see [use automated ML in
21693

21794
[!INCLUDE [list-of-inference-prebuilt-docker-images](../../includes/aml-inference-list-prebuilt-docker-images.md)]
21895

219-
## Security
96+
## Support
22097
Version updates for supported environments are released every two weeks to address vulnerabilities no older than 30 days.
22198

0 commit comments

Comments
 (0)