Skip to content

Commit 687cb9f

Browse files
edebar01edebar01
authored
Update how-to-troubleshoot-environments.md
1 parent 2a079e9 commit 687cb9f

File tree

1 file changed

+239
-31
lines changed

1 file changed

+239
-31
lines changed

‎articles/machine-learning/how-to-troubleshoot-environments.md

Lines changed: 239 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -690,6 +690,7 @@ from azureml.core.environment import CondaDependencies
690690
myenv = Environment(name="myenv")
691691
conda_dep = CondaDependencies()
692692
conda_dep.add_conda_package("python==3.8")
693+
env.python.conda_dependencies = conda_dep
693694
```
694695

695696
*Applies to: all scenarios*
@@ -826,7 +827,7 @@ from azureml.core.environment import CondaDependencies
826827
env = Environment(name="env")
827828
conda_dep = CondaDependencies()
828829
conda_dep.add_conda_package("python==3.8")
829-
env.python.conda_dependencies=conda_dep
830+
env.python.conda_dependencies = conda_dep
830831
```
831832

832833
*Applies to: Azure CLI & Python SDK v2*
@@ -881,7 +882,7 @@ from azureml.core.environment import CondaDependencies
881882
env = Environment(name="env")
882883
conda_dep = CondaDependencies()
883884
conda_dep.add_conda_package("python==3.8")
884-
env.python.conda_dependencies=conda_dep
885+
env.python.conda_dependencies = conda_dep
885886
```
886887

887888
*Applies to: Azure CLI & Python SDK v2*
@@ -897,50 +898,257 @@ You must specify a base Docker image for the environment, and the conda environm
897898
* See [how to set a conda specification on the environment definition](https://aka.ms/azureml/environment/set-conda-spec-on-environment-definition)
898899

899900
### Missing conda channels
900-
- If no conda channels are specified, conda will use defaults that might change
901-
- For reproducibility of your environment, specify channels from which to pull dependencies
902-
- For more information, see [how to manage conda channels](https://aka.ms/azureml/environment/managing-conda-channels)
901+
<!--issueDescription-->
902+
**Potential causes:**
903+
* You haven't specified conda channels in your environment definition
904+
905+
**Affected areas (symptoms):**
906+
* Failure in registering your environment
907+
<!--/issueDescription-->
908+
909+
**Troubleshooting steps**
910+
911+
For reproducibility of your environment, specify channels from which to pull dependencies. If no conda channel is specified, conda will use defaults that might change.
912+
913+
*Applies to: Python SDK v1*
914+
915+
Add a conda channel using the Python SDK
916+
917+
```python
918+
from azureml.core.environment import CondaDependencies
919+
920+
env = Environment(name="env")
921+
conda_dep = CondaDependencies()
922+
conda_dep.add_channel("conda-forge")
923+
env.python.conda_dependencies = conda_dep
924+
```
925+
926+
*Applies to: all scenarios*
927+
928+
If you're using a YAML for your conda specification, include the conda channel(s) you'd like to use
929+
930+
```yaml
931+
name: project_environment
932+
dependencies:
933+
- python=3.8
934+
- pip:
935+
- azureml-defaults
936+
channels:
937+
- anaconda
938+
- conda-forge
939+
```
940+
941+
**Resources**
942+
* See [how to set a conda specification on the environment definition v1](https://aka.ms/azureml/environment/set-conda-spec-on-environment-definition)
943+
* See [CondaDependencies class](https://aka.ms/azureml/environment/conda-dependencies-class)
944+
* See how to [create an environment from a conda specification v2](https://aka.ms/azureml/environment/create-env-conda-spec-v2)
945+
* See [how to create a conda file manually](https://aka.ms/azureml/environment/how-to-create-conda-file)
903946

904947
### Base conda environment not recommended
905-
- Partial environment updates can lead to dependency conflicts and/or unexpected runtime errors,
906-
so the use of base conda environments isn't recommended
907-
- Instead, specify all packages needed for your environment in the `conda_dependencies` section of your
908-
environment definition
909-
- See [from_conda_specification](https://aka.ms/azureml/environment/set-conda-spec-on-environment-definition)
910-
- See [CondaDependencies class](https://aka.ms/azureml/environment/conda-dependencies-class)
911-
- If you're using V2, add a conda specification to your [build context](https://aka.ms/azureml/environment/environment-build-context)
948+
<!--issueDescription-->
949+
**Potential causes:**
950+
* You specified a base conda environment in your environment definition
951+
952+
**Affected areas (symptoms):**
953+
* Failure in registering your environment
954+
<!--/issueDescription-->
955+
956+
**Troubleshooting steps**
957+
958+
Partial environment updates can lead to dependency conflicts and/or unexpected runtime errors, so the use of base conda environments isn't recommended.
959+
960+
*Applies to: Python SDK v1*
961+
962+
Remove your base conda environment, and specify all packages needed for your environment in the `conda_dependencies` section of your environment definition
963+
964+
```python
965+
from azureml.core.environment import CondaDependencies
966+
967+
env = Environment(name="env")
968+
env.python.base_conda_environment = None
969+
conda_dep = CondaDependencies()
970+
conda_dep.add_conda_package("python==3.8")
971+
env.python.conda_dependencies = conda_dep
972+
```
973+
974+
*Applies to: Azure CLI & Python SDK v2*
975+
976+
Define an environment using a standard conda YAML configuration file
977+
* See [how to create an environment from a conda specification](https://aka.ms/azureml/environment/create-env-conda-spec-v2)
978+
979+
**Resources**
980+
* See [how to set a conda specification on the environment definition v1](https://aka.ms/azureml/environment/set-conda-spec-on-environment-definition)
981+
* See [CondaDependencies class](https://aka.ms/azureml/environment/conda-dependencies-class)
982+
* See [how to create a conda file manually](https://aka.ms/azureml/environment/how-to-create-conda-file)
912983

913984
### Unpinned dependencies
914-
- For reproducibility, specify dependency versions for the packages in your conda specification
915-
- If versions aren't specified, there's a chance that the conda or pip package resolver will choose a different
916-
version of a package on subsequent builds of an environment. This behavior can lead to unexpected errors
917-
- See [conda package pinning](https://aka.ms/azureml/environment/how-to-pin-conda-packages)
985+
<!--issueDescription-->
986+
**Potential causes:**
987+
* You didn't specify versions for certain packages in your conda specification
988+
989+
**Affected areas (symptoms):**
990+
* Failure in registering your environment
991+
<!--/issueDescription-->
992+
993+
**Troubleshooting steps**
994+
995+
If a dependency version isn't specified, the conda package resolver may choose a different version of the package on subsequent builds of the same environment. This breaks reproducibility of the environment and can lead to unexpected errors.
996+
997+
*Applies to: Python SDK v1*
998+
999+
Include version numbers when adding packages to your conda specification
1000+
1001+
```python
1002+
from azureml.core.environment import CondaDependencies
1003+
1004+
conda_dep = CondaDependencies()
1005+
conda_dep.add_conda_package("numpy==1.24.1")
1006+
```
1007+
1008+
*Applies to: all scenarios*
1009+
1010+
If you're using a YAML for your conda specification, specify versions for your dependencies
1011+
1012+
```yaml
1013+
name: project_environment
1014+
dependencies:
1015+
- python=3.8
1016+
- pip:
1017+
- numpy=1.24.1
1018+
channels:
1019+
- anaconda
1020+
- conda-forge
1021+
```
1022+
1023+
**Resources**
1024+
* See [conda package pinning](https://aka.ms/azureml/environment/how-to-pin-conda-packages)
9181025

9191026
## *Pip issues*
9201027
### Pip not specified
921-
- For reproducibility, pip should be specified as a dependency in your conda specification, and it should be pinned
922-
- See [how to set a conda dependency](https://aka.ms/azureml/environment/add-conda-package-v1)
1028+
<!--issueDescription-->
1029+
**Potential causes:**
1030+
* You didn't specify pip as a dependency in your conda specification
1031+
1032+
**Affected areas (symptoms):**
1033+
* Failure in registering your environment
1034+
<!--/issueDescription-->
1035+
1036+
**Troubleshooting steps**
1037+
1038+
For reproducibility, pip should be specified as a dependency in your conda specification, and it should be pinned.
1039+
1040+
*Applies to: Python SDK v1*
1041+
1042+
Specify pip as a dependency, along with its version
1043+
1044+
```python
1045+
env.python.conda_dependencies.add_conda_package("pip==22.3.1")
1046+
```
1047+
1048+
*Applies to: all scenarios*
1049+
1050+
If you're using a YAML for your conda specification, specify pip as a dependency
1051+
1052+
```yaml
1053+
name: project_environment
1054+
dependencies:
1055+
- python=3.8
1056+
- pip=22.3.1
1057+
- pip:
1058+
- numpy=1.24.1
1059+
channels:
1060+
- anaconda
1061+
- conda-forge
1062+
```
1063+
1064+
**Resources**
1065+
* See [conda package pinning](https://aka.ms/azureml/environment/how-to-pin-conda-packages)
9231066

9241067
### Pip not pinned
925-
- For reproducibility, specify the pip resolver version in your conda dependencies
926-
- If the pip version isn't specified, there's a chance different versions of pip will be used on subsequent
927-
image builds on the environment
928-
- This behavior could cause the build to fail if the different pip versions resolve your packages differently
929-
- To avoid this issue and to achieve reproducibility of your environment, specify the pip version
930-
- See [conda package pinning](https://aka.ms/azureml/environment/how-to-pin-conda-packages)
931-
- See [how to set pip as a dependency](https://aka.ms/azureml/environment/add-conda-package-v1)
1068+
<!--issueDescription-->
1069+
**Potential causes:**
1070+
* You didn't specify a version for pip in your conda specification
1071+
1072+
**Affected areas (symptoms):**
1073+
* Failure in registering your environment
1074+
<!--/issueDescription-->
1075+
1076+
**Troubleshooting steps**
1077+
1078+
If a pip version isn't specified, a different version may be used on subsequent builds of the same environment. This can cause reproducibility issues and other unexpected errors if different versions of pip resolve your packages differently.
1079+
1080+
*Applies to: Python SDK v1*
1081+
1082+
Specify a pip version in your conda dependencies
1083+
1084+
```python
1085+
env.python.conda_dependencies.add_conda_package("pip==22.3.1")
1086+
```
1087+
1088+
*Applies to: all scenarios*
1089+
1090+
If you're using a YAML for your conda specification, specify a version for pip
1091+
1092+
```yaml
1093+
name: project_environment
1094+
dependencies:
1095+
- python=3.8
1096+
- pip=22.3.1
1097+
- pip:
1098+
- numpy=1.24.1
1099+
channels:
1100+
- anaconda
1101+
- conda-forge
1102+
```
1103+
1104+
**Resources**
1105+
* See [conda package pinning](https://aka.ms/azureml/environment/how-to-pin-conda-packages)
9321106

9331107
## *Deprecated environment property issues*
9341108
### R section is deprecated
935-
- The Azure Machine Learning SDK for R will be deprecated by the end of 2021 to make way for an improved R training and deployment
936-
experience using Azure Machine Learning CLI 2.0
937-
- See the [samples repository](https://aka.ms/azureml/environment/train-r-models-cli-v2) to get started with the edition CLI 2.0.
1109+
<!--issueDescription-->
1110+
**Potential causes:**
1111+
* You specified an R section in your environment definition
1112+
1113+
**Affected areas (symptoms):**
1114+
* Failure in registering your environment
1115+
<!--/issueDescription-->
1116+
1117+
**Troubleshooting steps**
1118+
1119+
The AzureML SDK for R was deprecated at the end of 2021 to make way for an improved R training and deployment experience using the Azure CLI v2
1120+
1121+
*Applies to: Python SDK v1*
1122+
1123+
Remove the R section from your environment definition
1124+
1125+
```python
1126+
env.r = None
1127+
```
1128+
1129+
*Applies to: all scenarios*
1130+
1131+
See the [samples repository](https://aka.ms/azureml/environment/train-r-models-cli-v2) to get started training R models using the Azure CLI v2
9381132

9391133
## **Image build problems**
9401134

9411135
## *Miscellaneous issues*
9421136
### Build log unavailable
943-
- Build logs are optional and not available for all environments since the image might already exist
1137+
<!--issueDescription-->
1138+
**Potential causes:**
1139+
* AzureML isn't authorized to store your build logs in your storage account
1140+
* A transient error occurred while saving your build logs
1141+
* Your image build didn't occur due to a system error before the build had a chance to start
1142+
1143+
**Affected areas (symptoms):**
1144+
* A successful build, but no available logs.
1145+
* Failure in building environments from UI, SDK, and CLI.
1146+
* Failure in running jobs because it will implicitly build the environment in the first step.
1147+
<!--/issueDescription-->
1148+
1149+
**Troubleshooting steps**
1150+
1151+
A rebuild may fix the issue if it's transient
9441152

9451153
## *ACR issues*
9461154
### ACR unreachable
@@ -954,8 +1162,8 @@ This issue can happen by failing to access a workspace's associated Azure Contai
9541162
**Affected areas (symptoms):**
9551163
* Failure in building environments from UI, SDK, and CLI.
9561164
* Failure in running jobs because it will implicitly build the environment in the first step.
957-
* Pipeline job failures
958-
* Model deployment failures
1165+
* Pipeline job failures.
1166+
* Model deployment failures.
9591167
<!--/issueDescription-->
9601168

9611169
**Troubleshooting steps**

0 commit comments

Comments
 (0)