Skip to content

Commit e942798

Browse files
authored
Merge pull request #228013 from santiagxf/santiagxf/mlflow-patch
Update how-to-use-mlflow-cli-runs.md
2 parents 4103d72 + a3029f8 commit e942798

File tree

1 file changed

+53
-32
lines changed

1 file changed

+53
-32
lines changed

articles/machine-learning/how-to-use-mlflow-cli-runs.md

Lines changed: 53 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -16,19 +16,44 @@ ms.devlang: azurecli
1616

1717
# Track ML experiments and models with MLflow
1818

19-
> [!div class="op_single_selector" title1="Select the version of Azure Machine Learning developer platform you are using:"]
19+
> [!div class="op_single_selector" title1="Select the version of Azure Machine Learning developer platform you're using:"]
2020
> * [v1](./v1/how-to-use-mlflow.md)
2121
> * [v2 (current version)](how-to-use-mlflow-cli-runs.md)
2222
23-
Azure Machine Learning workspaces are MLflow-compatible, which means you can use MLflow to track runs, metrics, parameters, and artifacts with your Azure Machine Learning workspaces. By using MLflow for tracking, you don't need to change your training routines to work with Azure Machine Learning or inject any cloud-specific syntax, which is one of the main advantages of the approach.
23+
__Tracking__ refers to process of saving all experiment's related information that you may find relevant for every experiment you run. Such metadata varies based on your project, but it may include:
2424

25-
See [MLflow and Azure Machine Learning](concept-mlflow.md) for all supported MLflow and Azure Machine Learning functionality including MLflow Project support (preview) and model deployment.
25+
> [!div class="checklist"]
26+
> - Code
27+
> - Environment details (OS version, Python packages)
28+
> - Input data
29+
> - Parameter configurations
30+
> - Models
31+
> - Evaluation metrics
32+
> - Evaluation visualizations (confusion matrix, importance plots)
33+
> - Evaluation results (including some evaluation predictions)
34+
35+
Some of these elements are automatically tracked by Azure Machine Learning when working with jobs (including code, environment, and input and output data). However, others like models, parameters, and metrics, need to be instrumented by the model builder as it's specific to the particular scenario.
2636

27-
In this article, you will learn how to use MLflow for tracking your experiments and runs in Azure Machine Learning workspaces.
37+
In this article, you'll learn how to use MLflow for tracking your experiments and runs in Azure Machine Learning workspaces.
2838

2939
> [!NOTE]
3040
> If you want to track experiments running on Azure Databricks or Azure Synapse Analytics, see the dedicated articles [Track Azure Databricks ML experiments with MLflow and Azure Machine Learning](how-to-use-mlflow-azure-databricks.md) or [Track Azure Synapse Analytics ML experiments with MLflow and Azure Machine Learning](how-to-use-mlflow-azure-synapse.md).
3141
42+
## Benefits of tracking experiments
43+
44+
We highly encourage machine learning practitioners to instrument their experimentation by tracking them, regardless if they're training with jobs in Azure Machine Learning or interactively in notebooks. Benefits include:
45+
46+
- All of your ML experiments are organized in a single place, allowing you to search and filter experiments to find the information and drill down to see what exactly it was that you tried before.
47+
- Compare experiments, analyze results, and debug model training with little extra work.
48+
- Reproduce or re-run experiments to validate results.
49+
- Improve collaboration by seeing what everyone is doing, sharing experiment results, and access experiment data programmatically.
50+
51+
### Why MLflow
52+
53+
Azure Machine Learning workspaces are MLflow-compatible, which means you can use MLflow to track runs, metrics, parameters, and artifacts with your Azure Machine Learning workspaces. By using MLflow for tracking, you don't need to change your training routines to work with Azure Machine Learning or inject any cloud-specific syntax, which is one of the main advantages of the approach.
54+
55+
See [MLflow and Azure Machine Learning](concept-mlflow.md) for all supported MLflow and Azure Machine Learning functionality including MLflow Project support (preview) and model deployment.
56+
3257
## Prerequisites
3358

3459
[!INCLUDE [mlflow-prereqs](../../includes/machine-learning-mlflow-prereqs.md)]
@@ -56,13 +81,13 @@ When submitting jobs using Azure Machine Learning CLI or SDK, you can set the ex
5681

5782
## Configure the run
5883

59-
Azure Machine Learning any training job in what MLflow calls a run. Use runs to capture all the processing that your job performs.
84+
Azure Machine Learning tracks any training job in what MLflow calls a run. Use runs to capture all the processing that your job performs.
6085

6186
# [Working interactively](#tab/interactive)
6287

63-
When working interactively, MLflow starts tracking your training routine as soon as you try to log information that requires an active run. For instance, when you log a metric, log a parameter, or when you start a training cycle when Mlflow's autologging functionality is enabled. However, it is usually helpful to start the run explicitly, specially if you want to capture the total time of your experiment in the field __Duration__. To start the run explicitly, use `mlflow.start_run()`.
88+
When working interactively, MLflow starts tracking your training routine as soon as you try to log information that requires an active run. For instance, when you log a metric, log a parameter, or when you start a training cycle when Mlflow's autologging functionality is enabled. However, it's usually helpful to start the run explicitly, specially if you want to capture the total time of your experiment in the field __Duration__. To start the run explicitly, use `mlflow.start_run()`.
6489

65-
Regardless if you started the run manually or not, you will eventually need to stop the run to inform MLflow that your experiment run has finished and marks its status as __Completed__. To do that, all `mlflow.end_run()`. We strongly recommend starting runs manually so you don't forget to end them when working on notebooks.
90+
Regardless if you started the run manually or not, you'll eventually need to stop the run to inform MLflow that your experiment run has finished and marks its status as __Completed__. To do that, all `mlflow.end_run()`. We strongly recommend starting runs manually so you don't forget to end them when working on notebooks.
6691

6792
```python
6893
mlflow.start_run()
@@ -72,7 +97,7 @@ mlflow.start_run()
7297
mlflow.end_run()
7398
```
7499

75-
To help you avoid forgetting to end the run, it is usually helpful to use the context manager paradigm:
100+
To help you avoid forgetting to end the run, it's usually helpful to use the context manager paradigm:
76101

77102
```python
78103
with mlflow.start_run() as run:
@@ -96,23 +121,23 @@ When working with jobs, you typically place all your training logic inside of a
96121

97122
:::code language="python" source="~/azureml-examples-main/cli/jobs/basics/src/hello-mlflow.py" highlight="9-10,12":::
98123

99-
The previous code example doesn't uses `mlflow.start_run()` but if used you can expect MLflow to reuse the current active run so there is no need to remove those lines if migrating to Azure Machine Learning.
124+
The previous code example doesn't uses `mlflow.start_run()` but if used you can expect MLflow to reuse the current active run so there's no need to remove those lines if migrating to Azure Machine Learning.
100125

101126
### Adding tracking to your routine
102127

103128
Use MLflow SDK to track any metric, parameter, artifacts, or models. For detailed examples about how to log each, see [Log metrics, parameters and files with MLflow](how-to-log-view-metrics.md).
104129

105130
### Ensure your job's environment has MLflow installed
106131

107-
All Azure Machine Learning environments already have MLflow installed for you, so no action is required if you are using a curated environment. If you want to use a custom environment:
132+
All Azure Machine Learning environments already have MLflow installed for you, so no action is required if you're using a curated environment. If you want to use a custom environment:
108133

109134
1. Create a `conda.yml` file with the dependencies you need:
110135

111136
:::code language="yaml" source="~/azureml-examples-main//sdk/python/using-mlflow/deploy/environment/conda.yml" highlight="7-8" range="1-12":::
112137

113-
1. Reference the environment in the job you are using.
138+
1. Reference the environment in the job you're using.
114139

115-
### Configuring the job's name
140+
### Configuring job's name
116141

117142
Use the parameter `display_name` of Azure Machine Learning jobs to configure the name of the run. The following example shows how:
118143

@@ -138,11 +163,11 @@ Use the parameter `display_name` of Azure Machine Learning jobs to configure the
138163
)
139164
```
140165

141-
2. Ensure you are not using `mlflow.start_run(run_name="")` inside of your training routine.
166+
2. Ensure you're not using `mlflow.start_run(run_name="")` inside of your training routine.
142167

143168
### Submitting the job
144169

145-
1. First, let's connect to Azure Machine Learning workspace where we are going to work on.
170+
1. First, let's connect to Azure Machine Learning workspace where we're going to work on.
146171

147172
# [Azure CLI](#tab/cli)
148173

@@ -211,40 +236,36 @@ The metrics and artifacts from MLflow logging are tracked in your workspace. To
211236

212237
:::image type="content" source="media/how-to-log-view-metrics/metrics.png" alt-text="Screenshot of the metrics view.":::
213238

214-
Select the logged metrics to render charts on the right side. You can customize the charts by applying smoothing, changing the color, or plotting multiple metrics on a single graph. You can also resize and rearrange the layout as you wish. Once you have created your desired view, you can save it for future use and share it with your teammates using a direct link.
239+
Select the logged metrics to render charts on the right side. You can customize the charts by applying smoothing, changing the color, or plotting multiple metrics on a single graph. You can also resize and rearrange the layout as you wish. Once you've created your desired view, you can save it for future use and share it with your teammates using a direct link.
215240

216-
Retrieve run metric using MLflow SDK, use [mlflow.get_run()](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.get_run).
241+
You can also access or __query metrics, parameters and artifacts programatically__ using the MLflow SDK. Use [mlflow.get_run()](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.get_run) as explained bellow:
217242

218-
```Python
219-
from mlflow.tracking import MlflowClient
243+
```python
244+
import mlflow
220245

221-
client = MlflowClient()
222-
run = MlflowClient().get_run("<RUN_ID>")
246+
run = mlflow.get_run("<RUN_ID>")
223247

224248
metrics = run.data.metrics
225-
tags = run.data.tags
226249
params = run.data.params
250+
tags = run.data.tags
227251

228-
print(metrics,tags,params)
252+
print(metrics, params, tags)
229253
```
230254

231-
To view the artifacts of a run, you can use [MlFlowClient.list_artifacts()](https://mlflow.org/docs/latest/python_api/mlflow.tracking.html#mlflow.tracking.MlflowClient.list_artifacts)
232-
233-
```Python
234-
client.list_artifacts(run_id)
235-
```
255+
> [!TIP]
256+
> For metrics, the previous example will only return the last value of a given metric. If you want to retrieve all the values of a given metric, use `mlflow.get_metric_history` method as explained at [Getting params and metrics from a run](how-to-track-experiments-mlflow.md#getting-params-and-metrics-from-a-run).
236257

237-
To download an artifact to the current directory, you can use [MLFlowClient.download_artifacts()](https://www.mlflow.org/docs/latest/python_api/mlflow.tracking.html#mlflow.tracking.MlflowClient.download_artifacts)
258+
To download artifacts you've logged, like files and models, you can use [mlflow.artifacts.download_artifacts()](https://www.mlflow.org/docs/latest/python_api/mlflow.artifacts.html#mlflow.artifacts.download_artifacts)
238259

239-
```Python
240-
client.download_artifacts(run_id, "helloworld.txt", ".")
260+
```python
261+
mlflow.artifacts.download_artifacts(run_id="<RUN_ID>", artifact_path="helloworld.txt")
241262
```
242263

243-
For more details about how to retrieve information from experiments and runs in Azure Machine Learning using MLflow view [Query & compare experiments and runs with MLflow](how-to-track-experiments-mlflow.md).
264+
For more details about how to __retrieve or compare__ information from experiments and runs in Azure Machine Learning using MLflow view [Query & compare experiments and runs with MLflow](how-to-track-experiments-mlflow.md)
244265

245266
## Example notebooks
246267

247-
If you are looking for examples about how to use MLflow in Jupyter notebooks, please see our example's repository [Using MLflow (Jupyter Notebooks)](https://github.com/Azure/azureml-examples/tree/main/sdk/python/using-mlflow).
268+
If you're looking for examples about how to use MLflow in Jupyter notebooks, please see our example's repository [Using MLflow (Jupyter Notebooks)](https://github.com/Azure/azureml-examples/tree/main/sdk/python/using-mlflow).
248269

249270
## Limitations
250271

0 commit comments

Comments
 (0)