Skip to content

Commit b8ece8b

Browse files
ttorblettorble
authored
Merge pull request #223263 from santiagxf/santiagxf/mlflow-projects
Update how-to-train-mlflow-projects.md
2 parents 45147bb + 9337541 commit b8ece8b

File tree

1 file changed

+125
-113
lines changed

1 file changed

+125
-113
lines changed

‎articles/machine-learning/how-to-train-mlflow-projects.md

Lines changed: 125 additions & 113 deletions
Original file line numberDiff line numberDiff line change
@@ -13,128 +13,139 @@ ms.topic: conceptual
1313
ms.custom: how-to, devx-track-python, sdkv2, event-tier1-build-2022
1414
---
1515

16-
# Train ML models with MLflow Projects and Azure Machine Learning (Preview)
16+
# Train with MLflow Projects in Azure Machine Learning (Preview)
1717

18-
In this article, learn how to enable MLflow's tracking URI and logging API, collectively known as [MLflow Tracking](https://mlflow.org/docs/latest/quickstart.html#using-the-tracking-api), to submit training jobs with [MLflow Projects](https://www.mlflow.org/docs/latest/projects.html) and Azure Machine Learning backend support. You can submit jobs locally with Azure Machine Learning tracking or migrate your runs to the cloud like via an [Azure Machine Learning Compute](./how-to-create-attach-compute-cluster.md).
18+
In this article, learn how to submit training jobs with [MLflow Projects](https://www.mlflow.org/docs/latest/projects.html) that uses Azure Machine Learning workspaces for tracking. You can submit jobs and only track them with Azure Machine Learning or migrate your runs to the cloud to run completely on [Azure Machine Learning Compute](./how-to-create-attach-compute-cluster.md).
1919

2020
[MLflow Projects](https://mlflow.org/docs/latest/projects.html) allow for you to organize and describe your code to let other data scientists (or automated tools) run it. MLflow Projects with Azure Machine Learning enable you to track and manage your training runs in your workspace.
2121

22-
[MLflow](https://www.mlflow.org) is an open-source library for managing the life cycle of your machine learning experiments. MLFlow Tracking is a component of MLflow that logs and tracks your training run metrics and model artifacts, no matter your experiment's environment--locally on your computer, on a remote compute target, a virtual machine, or an [Azure Databricks cluster](how-to-use-mlflow-azure-databricks.md).
23-
24-
[Learn more about the MLflow and Azure Machine Learning integration.](how-to-use-mlflow.md).
25-
26-
> [!TIP]
27-
> The information in this document is primarily for data scientists and developers who want to monitor the model training process. If you are an administrator interested in monitoring resource usage and events from Azure Machine Learning, such as quotas, completed training runs, or completed model deployments, see [Monitoring Azure Machine Learning](monitor-azure-machine-learning.md).
22+
[Learn more about the MLflow and Azure Machine Learning integration.](concept-mlflow.md)
2823

2924
## Prerequisites
3025

3126
[!INCLUDE [mlflow-prereqs](../../includes/machine-learning-mlflow-prereqs.md)]
3227

33-
### Connect to your workspace
34-
35-
First, let's connect MLflow to your Azure Machine Learning workspace.
36-
37-
# [Azure Machine Learning compute](#tab/aml)
38-
39-
Tracking is already configured for you. Your default credentials will also be used when working with MLflow.
40-
41-
# [Remote compute](#tab/remote)
42-
43-
**Configure tracking URI**
44-
45-
[!INCLUDE [configure-mlflow-tracking](../../includes/machine-learning-mlflow-configure-tracking.md)]
46-
47-
**Configure authentication**
48-
49-
Once the tracking is configured, you'll also need to configure how the authentication needs to happen to the associated workspace. By default, the Azure Machine Learning plugin for MLflow will perform interactive authentication by opening the default browser to prompt for credentials. Refer to [Configure MLflow for Azure Machine Learning: Configure authentication](how-to-use-mlflow-configure-tracking.md#configure-authentication) to additional ways to configure authentication for MLflow in Azure Machine Learning workspaces.
50-
51-
[!INCLUDE [configure-mlflow-auth](../../includes/machine-learning-mlflow-configure-auth.md)]
52-
53-
---
54-
55-
## Train MLflow Projects on local compute
56-
57-
This example shows how to submit MLflow projects locally with Azure Machine Learning.
58-
59-
Create the backend configuration object to store necessary information for the integration such as, the compute target and which type of managed environment to use.
28+
* Using Azure Machine Learning as backend for MLflow projects requires the package `azureml-core`:
6029

61-
```python
62-
backend_config = {"USE_CONDA": False}
63-
```
30+
```bash
31+
pip install azureml-core
32+
```
6433

65-
Add the `azureml-mlflow` package as a pip dependency to your environment configuration file in order to track metrics and key artifacts in your workspace.
66-
67-
``` shell
68-
name: mlflow-example
69-
channels:
70-
- defaults
71-
- anaconda
72-
- conda-forge
73-
dependencies:
74-
- python=3.6
75-
- scikit-learn=0.19.1
76-
- pip
77-
- pip:
78-
- mlflow
79-
- azureml-mlflow
80-
```
81-
82-
Submit the local run and ensure you set the parameter `backend = "azureml" `. With this setting, you can submit runs locally and get the added support of automatic output tracking, log files, snapshots, and printed errors in your workspace.
83-
84-
View your runs and metrics in the [Azure Machine Learning studio](https://ml.azure.com).
85-
86-
```python
87-
local_env_run = mlflow.projects.run(uri=".",
88-
parameters={"alpha":0.3},
89-
backend = "azureml",
90-
use_conda=False,
91-
backend_config = backend_config,
92-
)
93-
94-
```
95-
96-
## Train MLflow projects with remote compute
97-
98-
This example shows how to submit MLflow projects on a remote compute with Azure Machine Learning tracking.
99-
100-
Create the backend configuration object to store necessary information for the integration such as, the compute target and which type of managed environment to use.
101-
102-
The integration accepts "COMPUTE" and "USE_CONDA" as parameters where "COMPUTE" is set to the name of your remote compute cluster and "USE_CONDA" which creates a new environment for the project from the environment configuration file. If "COMPUTE" is present in the object, the project will be automatically submitted to the remote compute and ignore "USE_CONDA". MLflow accepts a dictionary object or a JSON file.
103-
104-
```python
105-
# dictionary
106-
backend_config = {"COMPUTE": "cpu-cluster", "USE_CONDA": False}
107-
```
108-
109-
Add the `azureml-mlflow` package as a pip dependency to your environment configuration file in order to track metrics and key artifacts in your workspace.
110-
111-
``` shell
112-
name: mlflow-example
113-
channels:
114-
- defaults
115-
- anaconda
116-
- conda-forge
117-
dependencies:
118-
- python=3.6
119-
- scikit-learn=0.19.1
120-
- pip
121-
- pip:
122-
- mlflow
123-
- azureml-mlflow
124-
```
125-
126-
Submit the mlflow project run and ensure you set the parameter `backend = "azureml" `. With this setting, you can submit your run to your remote compute and get the added support of automatic output tracking, log files, snapshots, and printed errors in your workspace.
127-
128-
View your runs and metrics in the [Azure Machine Learning studio](https://ml.azure.com).
129-
130-
```python
131-
remote_mlflow_run = mlflow.projects.run(uri=".",
132-
parameters={"alpha":0.3},
133-
backend = "azureml",
134-
backend_config = backend_config,
135-
)
34+
### Connect to your workspace
13635

137-
```
36+
If you're working outside Azure Machine Learning, you need to configure MLflow to point to your Azure Machine Learning workspace's tracking URI. You can find the instructions at [Configure MLflow for Azure Machine Learning](how-to-use-mlflow-configure-tracking.md).
37+
38+
39+
## Track MLflow Projects in Azure Machine Learning workspaces
40+
41+
This example shows how to submit MLflow projects and track them Azure Machine Learning.
42+
43+
1. Add the `azureml-mlflow` package as a pip dependency to your environment configuration file in order to track metrics and key artifacts in your workspace.
44+
45+
__conda.yaml__
46+
47+
```yaml
48+
name: mlflow-example
49+
channels:
50+
- defaults
51+
dependencies:
52+
- numpy>=1.14.3
53+
- pandas>=1.0.0
54+
- scikit-learn
55+
- pip:
56+
- mlflow
57+
- azureml-mlflow
58+
```
59+
60+
1. Submit the local run and ensure you set the parameter `backend = "azureml"`, which adds support of automatic tracking, model's capture, log files, snapshots, and printed errors in your workspace. In this example we assume the MLflow project you are trying to run is in the same folder you currently are, `uri="."`.
61+
62+
# [MLflow CLI](#tab/cli)
63+
64+
```bash
65+
mlflow run . --experiment-name --backend azureml --env-manager=local -P alpha=0.3
66+
```
67+
68+
# [Python](#tab/sdk)
69+
70+
```python
71+
local_env_run = mlflow.projects.run(
72+
uri=".",
73+
parameters={"alpha":0.3},
74+
backend = "azureml",
75+
env_manager="local",
76+
backend_config = backend_config,
77+
)
78+
```
79+
80+
---
81+
82+
View your runs and metrics in the [Azure Machine Learning studio](https://ml.azure.com).
83+
84+
## Train MLflow projects in Azure Machine Learning jobs
85+
86+
This example shows how to submit MLflow projects as a job running on Azure Machine Learning compute.
87+
88+
1. Create the backend configuration object, in this case we are going to indicate `COMPUTE`. This parameter references the name of your remote compute cluster you want to use for running your project. If `COMPUTE` is present, the project will be automatically submitted as an Azure Machine Learning job to the indicated compute.
89+
90+
# [MLflow CLI](#tab/cli)
91+
92+
__backend_config.json__
93+
94+
```json
95+
{
96+
"COMPUTE": "cpu-cluster"
97+
}
98+
99+
```
100+
101+
# [Python](#tab/sdk)
102+
103+
```python
104+
backend_config = {"COMPUTE": "cpu-cluster"}
105+
```
106+
107+
1. Add the `azureml-mlflow` package as a pip dependency to your environment configuration file in order to track metrics and key artifacts in your workspace.
108+
109+
__conda.yaml__
110+
111+
```yaml
112+
name: mlflow-example
113+
channels:
114+
- defaults
115+
dependencies:
116+
- numpy>=1.14.3
117+
- pandas>=1.0.0
118+
- scikit-learn
119+
- pip:
120+
- mlflow
121+
- azureml-mlflow
122+
```
123+
124+
1. Submit the local run and ensure you set the parameter `backend = "azureml"`, which adds support of automatic tracking, model's capture, log files, snapshots, and printed errors in your workspace. In this example we assume the MLflow project you are trying to run is in the same folder you currently are, `uri="."`.
125+
126+
# [MLflow CLI](#tab/cli)
127+
128+
```bash
129+
mlflow run . --backend azureml --backend-config backend_config.json -P alpha=0.3
130+
```
131+
132+
# [Python](#tab/sdk)
133+
134+
```python
135+
local_env_run = mlflow.projects.run(
136+
uri=".",
137+
parameters={"alpha":0.3},
138+
backend = "azureml",
139+
backend_config = backend_config,
140+
)
141+
```
142+
143+
---
144+
145+
> [!NOTE]
146+
> Since Azure Machine Learning jobs always run in the context of environments, the parameter `env_manager` is ignored.
147+
148+
View your runs and metrics in the [Azure Machine Learning studio](https://ml.azure.com).
138149

139150

140151
## Clean up resources
@@ -163,7 +174,8 @@ The [MLflow with Azure ML notebooks](https://github.com/Azure/MachineLearningNot
163174

164175
## Next steps
165176

166-
* [Deploy models with MLflow](how-to-deploy-mlflow-models.md).
167-
* Monitor your production models for [data drift](v1/how-to-enable-data-collection.md).
168177
* [Track Azure Databricks runs with MLflow](how-to-use-mlflow-azure-databricks.md).
169-
* [Manage your models](concept-model-management-and-deployment.md).
178+
* [Query & compare experiments and runs with MLflow](how-to-track-experiments-mlflow.md).
179+
* [Manage models registries in Azure Machine Learning with MLflow](how-to-manage-models-mlflow.md).
180+
* [Guidelines for deploying MLflow models](how-to-deploy-mlflow-models.md).
181+

0 commit comments

Comments
 (0)