Skip to content

Commit b8e1326

Browse files
authored
Create how-to-track-experiments-mlflow.md
1 parent 3b4916f commit b8e1326

File tree

1 file changed

+18
-20
lines changed

1 file changed

+18
-20
lines changed

articles/machine-learning/how-to-track-experiments-mlflow.md

Lines changed: 18 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -15,24 +15,24 @@ ms.custom: how-to, devx-track-python, ignite-2022
1515

1616
# Query & compare experiments and runs with MLflow
1717

18-
Experiments and runs tracking information in Azure Machine Learning can be queried using MLflow. You don't need to install any specific SDK to manage what happens inside of a training job, creating a more seamless transition between local runs and the cloud by removing cloud-specific dependencies. In this article, you'll learn how to query and compare experiments and runs in your workspace using Azure Machine Learning and MLflow SDK in Python.
18+
Experiments and runs tracking information in Azure Machine Learning can be queried using MLflow. You don't need to install any specific SDK to manage what happens inside of a training job, creating a more seamless transition between local runs and the cloud by removing cloud-specific dependencies.
19+
20+
> [!NOTE]
21+
> The Azure Machine Learning Python SDK v2 does not provide native logging or tracking capabilities. This applies not just for logging but also for querying the metrics logged. Instead, use MLflow to manage experiments and runs. This article explains how to use MLflow to manage experiments and runs in Azure Machine Learning.
1922
2023
MLflow allows you to:
2124

2225
* Create, query, delete and search for experiments in a workspace.
2326
* Query, delete, and search for runs in a workspace.
2427
* Track and retrieve metrics, parameters, artifacts and models from runs.
2528

26-
See [Support matrix for querying runs and experiments in Azure Machine Learning](#support-matrix-for-querying-runs-and-experiments) for a detailed comparison between MLflow Open-Source and MLflow when connected to Azure Machine Learning.
27-
28-
> [!NOTE]
29-
> The Azure Machine Learning Python SDK v2 does not provide native logging or tracking capabilities. This applies not just for logging but also for querying the metrics logged. Instead, use MLflow to manage experiments and runs. This article explains how to use MLflow to manage experiments and runs in Azure Machine Learning.
29+
In this article, you'll learn how to query and compare experiments and runs in your workspace using Azure Machine Learning and MLflow SDK in Python. See [Support matrix for querying runs and experiments in Azure Machine Learning](#support-matrix-for-querying-runs-and-experiments) for a detailed comparison between MLflow Open-Source and MLflow when connected to Azure Machine Learning.
3030

3131
### Prerequisites
3232

3333
[!INCLUDE [mlflow-prereqs](../../includes/machine-learning-mlflow-prereqs.md)]
3434

35-
## Get all the experiments
35+
## Getting all the experiments
3636

3737
You can get all the active experiments in the workspace using MLFlow:
3838

@@ -57,15 +57,13 @@ for exp in experiments:
5757

5858
## Search experiments
5959

60-
The `search_experiments()` method available since Mlflow 2.0 allows searching experiment matching a criteria using `filter_string`. The following query retrieves three experiments with different IDs.
60+
The `search_experiments()` method available since Mlflow 2.0 allows to search experiment matching a criteria using `filter_string`. The following query retrives three experiments with differents IDs.
6161

6262
```python
63-
mlflow.search_experiments(
64-
filter_string="experiment_id IN ('CDEFG-1234-5678-90AB', '1234-5678-90AB-CDEFG', '5678-1234-90AB-CDEFG')"
65-
)
63+
mlflow.search_experiments(filter_string="experiment_id IN ('CDEFG-1234-5678-90AB', '1234-5678-90AB-CDEFG', '5678-1234-90AB-CDEFG')")
6664
```
6765

68-
## Get a specific experiment
66+
## Getting a specific experiment
6967

7068
Details about a specific experiment can be retrieved using the `get_experiment_by_name` method:
7169

@@ -82,7 +80,7 @@ MLflow allows searching runs inside of any experiment, including multiple experi
8280
- Parameters with column's name `params.<parameter-name>`.
8381
- Metrics (last logged value of each) with column's name `metrics.<metric-name>`.
8482

85-
### Get all the runs from an experiment
83+
### Getting all the runs from an experiment
8684

8785
By experiment name:
8886

@@ -101,7 +99,7 @@ mlflow.search_runs(experiment_ids=[ "1234-5678-90AB-CDEFG" ])
10199
102100
Another important point to notice is that get returning runs, all metrics are parameters are also returned for them. However, for metrics containing multiple values (for instance, a loss curve, or a PR curve), only the last value of the metric is returned. If you want to retrieve all the values of a given metric, uses `mlflow.get_metric_history` method.
103101

104-
### Order runs
102+
### Ordering runs
105103

106104
By default, experiments are ordered descending by `start_time`, which is the time the experiment was queue in Azure Machine Learning. However, you can change this default by using the parameter `order_by`.
107105

@@ -124,7 +122,7 @@ You can also order by metrics to know which run generated the best results:
124122
mlflow.search_runs(experiment_ids=[ "1234-5678-90AB-CDEFG" ]).sort_values("metrics.accuracy", ascending=False)
125123
```
126124

127-
### Filter runs
125+
### Filtering runs
128126

129127
You can also look for a run with a specific combination in the hyperparameters using the parameter `filter_string`. Use `params` to access run's parameters and `metrics` to access metrics logged in the run. MLflow supports expressions joined by the AND keyword (the syntax does not support OR):
130128

@@ -164,7 +162,7 @@ runs = mlflow.search_runs(experiment_ids=[ "1234-5678-90AB-CDEFG" ])
164162
runs[runs.status == "FINISHED"]
165163
```
166164

167-
## Get metrics, parameters, artifacts and models
165+
## Getting metrics, parameters, artifacts and models
168166

169167
The method `search_runs` returns a Pandas `Dataframe` containing a limited amount of information by default. You can get Python objects if needed, which may be useful to get details about them. Use the `output_format` parameter to control how output is returned:
170168

@@ -183,7 +181,7 @@ last_run = runs[-1]
183181
print("Last run ID:", last_run.info.run_id)
184182
```
185183

186-
### Get params and metrics from a run
184+
### Getting params and metrics from a run
187185

188186
When runs are returned using `output_format="list"`, you can easily access parameters using the key `data`:
189187

@@ -204,7 +202,7 @@ client = mlflow.tracking.MlflowClient()
204202
client.get_metric_history("1234-5678-90AB-CDEFG", "log_loss")
205203
```
206204

207-
### Get artifacts from a run
205+
### Getting artifacts from a run
208206

209207
Any artifact logged by a run can be queried by MLflow. Artifacts can't be access using the run object itself and the MLflow client should be used instead:
210208

@@ -224,7 +222,7 @@ file_path = mlflow.artifacts.download_artifacts(
224222
> [!NOTE]
225223
> __MLflow 2.0 advisory:__ In legacy versions of MLflow (<2.0), use the method `MlflowClient.download_artifacts()` instead.
226224
227-
### Get models from a run
225+
### Getting models from a run
228226

229227
Models can also be logged in the run and then retrieved directly from it. To retrieve it, you need to know the artifact's path where it is stored. The method `list_artifacats` can be used to find artifacts that are representing a model since MLflow models are always folders. You can download a model by indicating the path where the model is stored using the `download_artifact` method:
230228

@@ -250,7 +248,7 @@ model = mlflow.xgboost.load_model(f"runs:/{last_run.info.run_id}/{artifact_path}
250248
> [!TIP]
251249
> For query and loading models registered in the Model Registry, view [Manage models registries in Azure Machine Learning with MLflow](how-to-manage-models-mlflow.md).
252250
253-
## Get child (nested) runs
251+
## Getting child (nested) runs
254252

255253
MLflow supports the concept of child (nested) runs. They are useful when you need to spin off training routines requiring being tracked independently from the main training process. Hyper-parameter tuning optimization processes or Azure Machine Learning pipelines are typical examples of jobs that generate multiple child runs. You can query all the child runs of a specific run using the property tag `mlflow.parentRunId`, which contains the run ID of the parent run.
256254

@@ -302,7 +300,7 @@ The MLflow SDK exposes several methods to retrieve runs, including options to co
302300
| Renaming experiments | **&check;** | |
303301

304302
> [!NOTE]
305-
> - <sup>1</sup> Check the section [Getting runs inside an experiment](#getting-runs-inside-an-experiment) for instructions and examples on how to achieve the same functionality in Azure Machine Learning.
303+
> - <sup>1</sup> Check the section [Query runs inside an experiment](#query-runs-inside-an-experiment) for instructions and examples on how to achieve the same functionality in Azure Machine Learning.
306304
> - <sup>2</sup> `!=` for tags not supported.
307305
308306
## Next steps

0 commit comments

Comments
 (0)