Skip to content

Commit fda6ef1

Browse files
committed
PM feedback + peer review
1 parent a8b0187 commit fda6ef1

File tree

1 file changed

+40
-31
lines changed

1 file changed

+40
-31
lines changed

articles/machine-learning/how-to-use-mlflow-cli-runs.md

Lines changed: 40 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -8,48 +8,49 @@ ms.author: osomorog
88
ms.service: machine-learning
99
ms.subservice: mlops
1010
ms.reviewer: nibaccam
11-
ms.date: 11/23/2021
11+
ms.date: 12/16/2021
1212
ms.topic: how-to
1313
ms.custom: devx-track-python
1414
---
1515

16-
# Track ML experiments and models with MLflow and Azure Machine Learning CLI (v2) (preview)
16+
# Track ML experiments and models with MLflow or Azure Machine Learning CLI (v2) (preview)
1717

1818
[!INCLUDE [preview disclaimer](../../includes/machine-learning-preview-generic-disclaimer.md)]
1919

20-
In this article, learn how to enable MLflow's tracking URI and logging API, collectively known as [MLflow Tracking](https://mlflow.org/docs/latest/quickstart.html#using-the-tracking-api), to connect Azure Machine Learning as the backend of your MLflow experiments with the [Azure Machine Learning CLI (v2)](how-to-train-cli.md). You also learn how to use [MLflow's Model Registry](https://mlflow.org/docs/latest/model-registry.html) capabilities with Azure Machine Learning.
20+
In this article, learn how to enable MLflow's tracking URI and logging API, collectively known as [MLflow Tracking](https://mlflow.org/docs/latest/quickstart.html#using-the-tracking-api), to connect Azure Machine Learning as the backend of your MLflow experiments. You can accomplish this connection with either the MLflow Python API or the [Azure Machine Learning CLI (v2)](how-to-train-cli.md). You also learn how to use [MLflow's Model Registry](https://mlflow.org/docs/latest/model-registry.html) capabilities with Azure Machine Learning.
2121

2222
[MLflow](https://www.mlflow.org) is an open-source library for managing the lifecycle of your machine learning experiments. MLFlow Tracking is a component of MLflow that logs and tracks your training run metrics and model artifacts, no matter your experiment's environment--locally on your computer, on a remote compute target, a virtual machine, or an [Azure Databricks cluster](how-to-use-mlflow-azure-databricks.md).
2323

2424
See [MLflow and Azure Machine Learning](concept-mlflow.md) for all supported MLflow and Azure Machine Learning functionality including MLflow Project support (preview) and model deployment.
2525

26+
2627
> [!TIP]
2728
> The information in this document is primarily for data scientists and developers who want to monitor the model training process. If you are an administrator interested in monitoring resource usage and events from Azure Machine Learning, such as quotas, completed training runs, or completed model deployments, see [Monitoring Azure Machine Learning](monitor-azure-machine-learning.md).
2829
29-
> [!NOTE]
30+
> [!NOTE]
3031
> You can use the [MLflow Skinny client](https://github.com/mlflow/mlflow/blob/master/README_SKINNY.rst) which is a lightweight MLflow package without SQL storage, server, UI, or data science dependencies. This is recommended for users who primarily need the tracking and logging capabilities without importing the full suite of MLflow features including deployments.
3132
3233
## Prerequisites
3334

34-
* Install the `azureml-mlflow` package.
35-
* This package automatically brings in `azureml-core` of the [The Azure Machine Learning Python SDK](/python/api/overview/azure/ml/install), which provides the connectivity for MLflow to access your workspace.
36-
3735
* [Create an Azure Machine Learning Workspace](how-to-manage-workspace.md).
3836
* See which [access permissions you need to perform your MLflow operations with your workspace](how-to-assign-roles.md#mlflow-operations).
3937

40-
* Install and [set up CLI (v2)](how-to-configure-cli.md#prerequisites).
38+
* Install and [set up CLI (v2)](how-to-configure-cli.md#prerequisites) and make sure you install the ml extension
4139

4240
## Track runs from your local machine
4341

4442
MLflow Tracking with Azure Machine Learning lets you store the logged metrics and artifacts runs that were executed on your local machine into your Azure Machine Learning workspace.
4543

46-
### Setting up tracking environment to track local run
44+
### Set up tracking environment
4745

48-
First, you need to point your local machine to the Azure Machine Learning MLFlow Tracking URI.
46+
To track a local run, you need to point your local machine to the Azure Machine Learning MLFlow Tracking URI.
4947

50-
This can be done in a couple ways. You can import the `mlflow` and [`subprocess`](https://docs.python.org/3/library/subprocess.html) classes to access MLflow's tracking URI.
48+
>[!NOTE]
49+
>The tracking URI is valid up to an hour or less. If you restart your script after some idle time, use the get_mlflow_tracking_uri API to get a new URI.
5150
52-
The following code, uses the subprocess class in Python to run the Azure Machine Learning CLI (v2) command to retrieve the unique MLFLow tracking URI associated with your workspace, and [`set_tracking_uri()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.set_tracking_uri) points the MLflow tracking URI to that URI.
51+
# [MLflow](#tab/mlflow)
52+
53+
The following code uses `mlflow` and the [`subprocess`](https://docs.python.org/3/library/subprocess.html) classes in Python to run the Azure Machine Learning CLI (v2) command to retrieve the unique MLFLow tracking URI associated with your workspace. Then the method [`set_tracking_uri()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.set_tracking_uri) points the MLflow tracking URI to that URI.
5354

5455
```Python
5556
import mlflow
@@ -60,41 +61,49 @@ MLFLOW_TRACKING_URI = subprocess.run(["az", "ml", "workspace", "show", "--query"
6061
MLFLOW_TRACKING_URI = str(MLFLOW_TRACKING_URI.stdout).strip()
6162
```
6263

64+
# [Azure CLI](#tab/azure-cli)
65+
6366
Another option is to set one of the MLflow environment variables [MLFLOW_TRACKING_URI](https://mlflow.org/docs/latest/tracking.html#logging-to-a-tracking-server) directly in your terminal.
6467

65-
```Bash
68+
```Azure CLI
6669
# Configure MLflow to communicate with a Azure Machine Learning-hosted tracking server
70+
6771
export MLFLOW_TRACKING_URI=$(az ml workspace show --query mlflow_tracking_uri | sed 's/"//g')
6872
```
73+
---
6974

70-
>[!NOTE]
71-
>The tracking URI is valid up to an hour or less. If you restart your script after some idle time, use the get_mlflow_tracking_uri API to get a new URI.
7275

73-
Next, you set the experiment name.
76+
### Set experiment name
7477

75-
All MLflow runs are logged to the active experiment, which can be set using any of the following ways:
78+
All MLflow runs are logged to the active experiment, which can be set using MLflow or the Azure CLI.
7679

77-
* You can use the [mlflow.set_experiment()](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.set_experiment) command.
80+
# [MLflow](#tab/mlflow)
81+
82+
With MLflow you can use the [`mlflow.set_experiment()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.set_experiment) command.
7883

7984
```Python
8085
experiment_name = 'experiment_with_mlflow'
8186
mlflow.set_experiment(experiment_name)
8287
```
8388

84-
* You can use the `experiment_id` parameter in the [mlflow.start_run()](https://www.mlflow.org/docs/latest/python_api/mlflow.html#mlflow.start_run) command.
89+
Another option is to use the `experiment_id` parameter in the [mlflow.start_run()](https://www.mlflow.org/docs/latest/python_api/mlflow.html#mlflow.start_run) command.
8590

8691
```Python
8792
experiment_name = 'experiment_with_mlflow'
8893
with mlflow.start_run(experiment_id=experiment_name) as mlflow_run:
8994
mlflow.log_param("hello_param", "world")
9095
```
96+
# [Azure CLI](#tab/azure-cli)
9197

92-
* Another option is to set one of the MLflow environment variables [MLFLOW_EXPERIMENT_NAME or MLFLOW_EXPERIMENT_ID](https://mlflow.org/docs/latest/cli.html#cmdoption-mlflow-run-arg-uri).
98+
With the Azure CLI, set one of the MLflow environment variables [MLFLOW_EXPERIMENT_NAME or MLFLOW_EXPERIMENT_ID](https://mlflow.org/docs/latest/cli.html#cmdoption-mlflow-run-arg-uri) with the experiment name.
9399

94-
```Bash
100+
```Azure CLI
95101
# Configure MLflow to communicate with a Azure Machine Learning-hosted tracking server
96102
export MLFLOW_EXPERIMENT_NAME="experiment_with_mlflow"
97103
```
104+
---
105+
106+
### Start training run
98107

99108
After you set the MLflow experiment name, you can start your training run with `start_run()`. Then use `log_metric()` to activate the MLflow logging API and begin logging your training run metrics.
100109

@@ -113,7 +122,7 @@ with mlflow.start_run() as mlflow_run:
113122

114123
Remote runs (jobs) let you train your models on more powerful computes, such as GPU enabled virtual machines, or Machine Learning Compute clusters. See [Use compute targets for model training](how-to-set-up-training-targets.md) to learn about different compute options.
115124

116-
MLflow Tracking with Azure Machine Learning lets you store the logged metrics and artifacts from your remote runs into your Azure Machine Learning workspace. Any run with MLflow Tracking code in it will have metrics logged automatically to the workspace.
125+
MLflow Tracking with Azure Machine Learning lets you store the logged metrics and artifacts from your remote runs into your Azure Machine Learning workspace. Any run with MLflow Tracking code in it logs metrics automatically to the workspace.
117126

118127
First, you should create a `src` subdirectory and create a file with your training code in a `train.py` file in the `src` subdirectory. All your training code will go into the `src` subdirectory, including `train.py`.
119128

@@ -142,7 +151,7 @@ if __name__ == "__main__":
142151
main()
143152
```
144153

145-
Use the [Azure Machine Learning CLI (v2)](how-to-train-cli.md) to submit a run. When using the Azure Machine Learning CLI (v2), the MLflow tracking URI and experiment name is set automatically and directs the logging from MLflow to your Workspace. Learn more about [logging Azure Machine Learning CLI (v2) experiments with MLflow](how-to-train-cli.md#model-tracking-with-mlflow)
154+
Use the [Azure Machine Learning CLI (v2)](how-to-train-cli.md) to submit a run. When using the Azure Machine Learning CLI (v2), the MLflow tracking URI and experiment name are set automatically and directs the logging from MLflow to your workspace. Learn more about [logging Azure Machine Learning CLI (v2) experiments with MLflow](how-to-train-cli.md#model-tracking-with-mlflow)
146155

147156
Create a YAML file with your job definition in a `job.yml` file. This file should be created outside the `src` directory. Copy this code into the file:
148157

@@ -162,7 +171,7 @@ compute: azureml:MyCluster
162171
163172
Open your terminal and use the following to submit the job.
164173
165-
```Bash
174+
```Azure CLI
166175
az ml job create -f job.yml --web
167176
```
168177

@@ -172,7 +181,7 @@ The metrics and artifacts from MLflow logging are tracked in your workspace. To
172181

173182
Retrieve run metric using MLFlow [get_run()](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.get_run).
174183

175-
```python
184+
```Python
176185
from mlflow.tracking import MlflowClient
177186
# Use MlFlow to retrieve the run that was just completed
178187
client = MlflowClient()
@@ -189,21 +198,22 @@ print(metrics,tags,params)
189198
### Retrieve artifacts with MLFLow
190199

191200
To view the artifacts of a run, you can use [MlFlowClient.list_artifacts()](https://mlflow.org/docs/latest/python_api/mlflow.tracking.html#mlflow.tracking.MlflowClient.list_artifacts)
192-
```python
201+
202+
```Python
193203
client.list_artifacts(run_id)
194204
```
195205

196206
To download an artifact to the current directory, you can use [MLFlowClient.download_artifacts()](https://www.mlflow.org/docs/latest/python_api/mlflow.tracking.html#mlflow.tracking.MlflowClient.download_artifacts)
197207

198-
```python
208+
```Python
199209
client.download_artifacts(run_id, "helloworld.txt", ".")
200210
```
201211

202212
### Compare and query
203213

204214
To compare and query all MLFlow runs in your Azure Machine Learning Workspace
205215

206-
```python
216+
```Python
207217
from mlflow.entities import ViewType
208218

209219
all_experiments = [exp.experiment_id for exp in MlflowClient().list_experiments()]
@@ -223,7 +233,7 @@ To register and view a model from a run, use the following steps:
223233

224234
1. Once a run is complete, call the [`register_model()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.register_model) method.
225235

226-
```python
236+
```Python
227237
# the model folder produced from a run is registered. This includes the MLmodel file, model.pkl and the conda.yaml.
228238
model_path = "model"
229239
model_uri = 'runs:/{}/{}'.format(run_id, model_path)
@@ -247,8 +257,7 @@ To register and view a model from a run, use the following steps:
247257

248258
## Example notebooks
249259

250-
A community-driven repository of examples using mlflow can be found at https://github.com/Azure/azureml-examples.
251-
* [Using MLFlow and CLI (v2)](https://github.com/Azure/azureml-examples/blob/main/cli/jobs/basics/hello-mlflow.yml)
260+
[Use MLFlow and CLI (v2)](https://github.com/Azure/azureml-examples/blob/main/cli/jobs/basics/hello-mlflow.yml)
252261

253262
## Next steps
254263

0 commit comments

Comments
 (0)