You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Azure Machine Learning workspaces are MLflow-compatible, which means you can use MLflow to track runs, metrics, parameters, and artifacts with your Azure Machine Learning workspaces. By using MLflow for tracking, you don't need to change your training routines to work with Azure Machine Learning or inject any cloud-specific syntax, which is one of the main advantages of the approach.
23
+
__Tracking__ refers to process of saving all experiment's related information that you may find relevant for every experiment you run. Such metadata varies based on your project, but it may include:
24
24
25
-
See [MLflow and Azure Machine Learning](concept-mlflow.md) for all supported MLflow and Azure Machine Learning functionality including MLflow Project support (preview) and model deployment.
> - Evaluation results (including some evaluation predictions)
34
+
35
+
Some of these elements are automatically tracked by Azure Machine Learning when working with jobs (including code, environment, and input and output data). However, others like models, parameters, and metrics, need to be instrumented by the model builder as it's specific to the particular scenario.
26
36
27
-
In this article, you will learn how to use MLflow for tracking your experiments and runs in Azure Machine Learning workspaces.
37
+
In this article, you'll learn how to use MLflow for tracking your experiments and runs in Azure Machine Learning workspaces.
28
38
29
39
> [!NOTE]
30
40
> If you want to track experiments running on Azure Databricks or Azure Synapse Analytics, see the dedicated articles [Track Azure Databricks ML experiments with MLflow and Azure Machine Learning](how-to-use-mlflow-azure-databricks.md) or [Track Azure Synapse Analytics ML experiments with MLflow and Azure Machine Learning](how-to-use-mlflow-azure-synapse.md).
31
41
42
+
## Benefits of tracking experiments
43
+
44
+
We highly encourage machine learning practitioners to instrument their experimentation by tracking them, regardless if they're training with jobs in Azure Machine Learning or interactively in notebooks. Benefits include:
45
+
46
+
- All of your ML experiments are organized in a single place, allowing you to search and filter experiments to find the information and drill down to see what exactly it was that you tried before.
47
+
- Compare experiments, analyze results, and debug model training with little extra work.
48
+
- Reproduce or re-run experiments to validate results.
49
+
- Improve collaboration by seeing what everyone is doing, sharing experiment results, and access experiment data programmatically.
50
+
51
+
### Why MLflow
52
+
53
+
Azure Machine Learning workspaces are MLflow-compatible, which means you can use MLflow to track runs, metrics, parameters, and artifacts with your Azure Machine Learning workspaces. By using MLflow for tracking, you don't need to change your training routines to work with Azure Machine Learning or inject any cloud-specific syntax, which is one of the main advantages of the approach.
54
+
55
+
See [MLflow and Azure Machine Learning](concept-mlflow.md) for all supported MLflow and Azure Machine Learning functionality including MLflow Project support (preview) and model deployment.
@@ -56,13 +81,13 @@ When submitting jobs using Azure Machine Learning CLI or SDK, you can set the ex
56
81
57
82
## Configure the run
58
83
59
-
Azure Machine Learning any training job in what MLflow calls a run. Use runs to capture all the processing that your job performs.
84
+
Azure Machine Learning tracks any training job in what MLflow calls a run. Use runs to capture all the processing that your job performs.
60
85
61
86
# [Working interactively](#tab/interactive)
62
87
63
-
When working interactively, MLflow starts tracking your training routine as soon as you try to log information that requires an active run. For instance, when you log a metric, log a parameter, or when you start a training cycle when Mlflow's autologging functionality is enabled. However, it is usually helpful to start the run explicitly, specially if you want to capture the total time of your experiment in the field __Duration__. To start the run explicitly, use `mlflow.start_run()`.
88
+
When working interactively, MLflow starts tracking your training routine as soon as you try to log information that requires an active run. For instance, when you log a metric, log a parameter, or when you start a training cycle when Mlflow's autologging functionality is enabled. However, it's usually helpful to start the run explicitly, specially if you want to capture the total time of your experiment in the field __Duration__. To start the run explicitly, use `mlflow.start_run()`.
64
89
65
-
Regardless if you started the run manually or not, you will eventually need to stop the run to inform MLflow that your experiment run has finished and marks its status as __Completed__. To do that, all `mlflow.end_run()`. We strongly recommend starting runs manually so you don't forget to end them when working on notebooks.
90
+
Regardless if you started the run manually or not, you'll eventually need to stop the run to inform MLflow that your experiment run has finished and marks its status as __Completed__. To do that, all `mlflow.end_run()`. We strongly recommend starting runs manually so you don't forget to end them when working on notebooks.
66
91
67
92
```python
68
93
mlflow.start_run()
@@ -72,7 +97,7 @@ mlflow.start_run()
72
97
mlflow.end_run()
73
98
```
74
99
75
-
To help you avoid forgetting to end the run, it is usually helpful to use the context manager paradigm:
100
+
To help you avoid forgetting to end the run, it's usually helpful to use the context manager paradigm:
76
101
77
102
```python
78
103
with mlflow.start_run() as run:
@@ -96,23 +121,23 @@ When working with jobs, you typically place all your training logic inside of a
The previous code example doesn't uses `mlflow.start_run()` but if used you can expect MLflow to reuse the current active run so there is no need to remove those lines if migrating to Azure Machine Learning.
124
+
The previous code example doesn't uses `mlflow.start_run()` but if used you can expect MLflow to reuse the current active run so there's no need to remove those lines if migrating to Azure Machine Learning.
100
125
101
126
### Adding tracking to your routine
102
127
103
128
Use MLflow SDK to track any metric, parameter, artifacts, or models. For detailed examples about how to log each, see [Log metrics, parameters and files with MLflow](how-to-log-view-metrics.md).
104
129
105
130
### Ensure your job's environment has MLflow installed
106
131
107
-
All Azure Machine Learning environments already have MLflow installed for you, so no action is required if you are using a curated environment. If you want to use a custom environment:
132
+
All Azure Machine Learning environments already have MLflow installed for you, so no action is required if you're using a curated environment. If you want to use a custom environment:
108
133
109
134
1. Create a `conda.yml` file with the dependencies you need:
1. Reference the environment in the job you are using.
138
+
1. Reference the environment in the job you're using.
114
139
115
-
### Configuring the job's name
140
+
### Configuring job's name
116
141
117
142
Use the parameter `display_name` of Azure Machine Learning jobs to configure the name of the run. The following example shows how:
118
143
@@ -138,11 +163,11 @@ Use the parameter `display_name` of Azure Machine Learning jobs to configure the
138
163
)
139
164
```
140
165
141
-
2. Ensure you arenot using `mlflow.start_run(run_name="")` inside of your training routine.
166
+
2. Ensure you're not using `mlflow.start_run(run_name="")` inside of your training routine.
142
167
143
168
### Submitting the job
144
169
145
-
1. First, let's connect to Azure Machine Learning workspace where we are going to work on.
170
+
1. First, let's connect to Azure Machine Learning workspace where we're going to work on.
146
171
147
172
# [Azure CLI](#tab/cli)
148
173
@@ -211,40 +236,36 @@ The metrics and artifacts from MLflow logging are tracked in your workspace. To
211
236
212
237
:::image type="content"source="media/how-to-log-view-metrics/metrics.png" alt-text="Screenshot of the metrics view.":::
213
238
214
-
Select the logged metrics to render charts on the right side. You can customize the charts by applying smoothing, changing the color, or plotting multiple metrics on a single graph. You can also resize and rearrange the layout as you wish. Once you have created your desired view, you can save it for future use and share it with your teammates using a direct link.
239
+
Select the logged metrics to render charts on the right side. You can customize the charts by applying smoothing, changing the color, or plotting multiple metrics on a single graph. You can also resize and rearrange the layout as you wish. Once you've created your desired view, you can save it for future use and share it with your teammates using a direct link.
215
240
216
-
Retrieve run metric using MLflow SDK, use [mlflow.get_run()](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.get_run).
241
+
You can also access or __query metrics, parameters and artifacts programatically__ using the MLflow SDK. Use [mlflow.get_run()](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.get_run) as explained bellow:
217
242
218
-
```Python
219
-
from mlflow.tracking importMlflowClient
243
+
```python
244
+
importmlflow
220
245
221
-
client= MlflowClient()
222
-
run= MlflowClient().get_run("<RUN_ID>")
246
+
run= mlflow.get_run("<RUN_ID>")
223
247
224
248
metrics= run.data.metrics
225
-
tags= run.data.tags
226
249
params= run.data.params
250
+
tags= run.data.tags
227
251
228
-
print(metrics,tags,params)
252
+
print(metrics,params, tags)
229
253
```
230
254
231
-
To view the artifacts of a run, you can use [MlFlowClient.list_artifacts()](https://mlflow.org/docs/latest/python_api/mlflow.tracking.html#mlflow.tracking.MlflowClient.list_artifacts)
232
-
233
-
```Python
234
-
client.list_artifacts(run_id)
235
-
```
255
+
> [!TIP]
256
+
> For metrics, the previous example will only return the last value of a given metric. If you want to retrieve all the values of a given metric, use `mlflow.get_metric_history` method as explained at [Getting params and metrics from a run](how-to-track-experiments-mlflow.md#getting-params-and-metrics-from-a-run).
236
257
237
-
To download an artifact to the current directory, you can use [MLFlowClient.download_artifacts()](https://www.mlflow.org/docs/latest/python_api/mlflow.tracking.html#mlflow.tracking.MlflowClient.download_artifacts)
258
+
To download artifacts you've logged, like files and models, you can use [mlflow.artifacts.download_artifacts()](https://www.mlflow.org/docs/latest/python_api/mlflow.artifacts.html#mlflow.artifacts.download_artifacts)
For more details about how to retrieve information from experiments and runs in Azure Machine Learning using MLflow view [Query & compare experiments and runs with MLflow](how-to-track-experiments-mlflow.md).
264
+
For more details about how to __retrieve or compare__ information from experiments and runs in Azure Machine Learning using MLflow view [Query & compare experiments and runs with MLflow](how-to-track-experiments-mlflow.md)
244
265
245
266
## Example notebooks
246
267
247
-
If you are looking for examples about how to use MLflow in Jupyter notebooks, please see our example's repository [Using MLflow (Jupyter Notebooks)](https://github.com/Azure/azureml-examples/tree/main/sdk/python/using-mlflow).
268
+
If you're looking for examples about how to use MLflow in Jupyter notebooks, please see our example's repository [Using MLflow (Jupyter Notebooks)](https://github.com/Azure/azureml-examples/tree/main/sdk/python/using-mlflow).
0 commit comments