Skip to content

Commit 788a565

Browse files
v-stsavellv-stsavell
authored
Merge pull request #204413 from santiagxf/santiagxf/aml-mlflow-projects
AML MLflow projects upgrade to V2
2 parents 437fe44 + 2fa13d9 commit 788a565

File tree

1 file changed

+83
-43
lines changed

1 file changed

+83
-43
lines changed

‎articles/machine-learning/how-to-train-mlflow-projects.md

Lines changed: 83 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,6 @@ ms.custom: how-to, devx-track-python, sdkv1, event-tier1-build-2022
1414

1515
# Train ML models with MLflow Projects and Azure Machine Learning (preview)
1616

17-
[!INCLUDE [sdk v1](../../includes/machine-learning-sdk-v1.md)]
18-
19-
2017
[!INCLUDE [preview disclaimer](../../includes/machine-learning-preview-generic-disclaimer.md)]
2118

2219
In this article, learn how to enable MLflow's tracking URI and logging API, collectively known as [MLflow Tracking](https://mlflow.org/docs/latest/quickstart.html#using-the-tracking-api), to submit training jobs with [MLflow Projects](https://www.mlflow.org/docs/latest/projects.html) and Azure Machine Learning backend support. You can submit jobs locally with Azure Machine Learning tracking or migrate your runs to the cloud like via an [Azure Machine Learning Compute](./how-to-create-attach-compute-cluster.md).
@@ -33,37 +30,104 @@ In this article, learn how to enable MLflow's tracking URI and logging API, coll
3330
## Prerequisites
3431

3532
* Install the `azureml-mlflow` package.
36-
* This package automatically brings in `azureml-core` of the [The Azure Machine Learning Python SDK](/python/api/overview/azure/ml/install), which provides the connectivity for MLflow to access your workspace.
3733
* [Create an Azure Machine Learning Workspace](quickstart-create-resources.md).
3834
* See which [access permissions you need to perform your MLflow operations with your workspace](how-to-assign-roles.md#mlflow-operations).
35+
* Configure MLflow for tracking in Azure Machine Learning, as explained in the next section.
3936

40-
## Train MLflow Projects on local compute
37+
### Set up tracking environment
38+
39+
To configure MLflow for working with Azure Machine Learning, you need to point your MLflow environment to the Azure Machine Learning MLflow Tracking URI.
40+
41+
> [!NOTE]
42+
> When running on Azure Compute (Azure Notebooks, Jupyter Notebooks hosted on Azure Compute Instances or Compute Clusters) you don't have to configure the tracking URI. It's automatically configured for you.
43+
44+
# [Using the Azure ML SDK v2](#tab/azuremlsdk)
45+
46+
[!INCLUDE [sdk v2](../../includes/machine-learning-sdk-v2.md)]
47+
48+
You can get the Azure ML MLflow tracking URI using the [Azure Machine Learning SDK v2 for Python](concept-v2.md). Ensure you have the library `azure-ai-ml` installed in the cluster you are using. The following sample gets the unique MLFLow tracking URI associated with your workspace. Then the method [`set_tracking_uri()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.set_tracking_uri) points the MLflow tracking URI to that URI.
49+
50+
1. Using the workspace configuration file:
51+
52+
```Python
53+
from azure.ai.ml import MLClient
54+
from azure.identity import DefaultAzureCredential
55+
import mlflow
4156

42-
This example shows how to submit MLflow projects locally with Azure Machine Learning tracking.
57+
ml_client = MLClient.from_config(credential=DefaultAzureCredential()
58+
azureml_mlflow_uri = ml_client.workspaces.get(ml_client.workspace_name).mlflow_tracking_uri
59+
mlflow.set_tracking_uri(azureml_mlflow_uri)
60+
```
4361

44-
Install the `azureml-mlflow` package to use MLflow Tracking with Azure Machine Learning on your experiments locally. Your experiments can run via a Jupyter Notebook or code editor.
62+
> [!TIP]
63+
> You can download the workspace configuration file by:
64+
> 1. Navigate to [Azure ML studio](https://ml.azure.com)
65+
> 2. Click on the uper-right corner of the page -> Download config file.
66+
> 3. Save the file `config.json` in the same directory where you are working on.
4567

46-
```shell
47-
pip install azureml-mlflow
68+
1. Using the subscription ID, resource group name and workspace name:
69+
70+
```Python
71+
from azure.ai.ml import MLClient
72+
from azure.identity import DefaultAzureCredential
73+
import mlflow
74+
75+
#Enter details of your AzureML workspace
76+
subscription_id = '<SUBSCRIPTION_ID>'
77+
resource_group = '<RESOURCE_GROUP>'
78+
workspace_name = '<AZUREML_WORKSPACE_NAME>'
79+
80+
ml_client = MLClient(credential=DefaultAzureCredential(),
81+
subscription_id=subscription_id,
82+
resource_group_name=resource_group)
83+
84+
azureml_mlflow_uri = ml_client.workspaces.get(workspace_name).mlflow_tracking_uri
85+
mlflow.set_tracking_uri(azureml_mlflow_uri)
86+
```
87+
88+
> [!IMPORTANT]
89+
> `DefaultAzureCredential` will try to pull the credentials from the available context. If you want to specify credentials in a different way, for instance using the web browser in an interactive way, you can use `InteractiveBrowserCredential` or any other method available in `azure.identity` package.
90+
91+
# [Using an environment variable](#tab/environ)
92+
93+
[!INCLUDE [cli v2](../../includes/machine-learning-cli-v2.md)]
94+
95+
Another option is to set one of the MLflow environment variables [MLFLOW_TRACKING_URI](https://mlflow.org/docs/latest/tracking.html#logging-to-a-tracking-server) directly in your terminal.
96+
97+
```Azure CLI
98+
export MLFLOW_TRACKING_URI=$(az ml workspace show --query mlflow_tracking_uri | sed 's/"//g')
4899
```
49100

50-
Import the `mlflow` and [`Workspace`](/python/api/azureml-core/azureml.core.workspace%28class%29) classes to access MLflow's tracking URI and configure your workspace.
101+
>[!IMPORTANT]
102+
> Make sure you are logged in to your Azure account on your local machine, otherwise the tracking URI returns an empty string. If you are using any Azure ML compute the tracking environment and experiment name is already configured.
103+
104+
# [Building the MLflow tracking URI](#tab/build)
51105

52-
```Python
106+
The Azure Machine Learning Tracking URI can be constructed using the subscription ID, region of where the resource is deployed, resource group name and workspace name. The following code sample shows how:
107+
108+
```python
53109
import mlflow
54-
from azureml.core import Workspace
55110

56-
ws = Workspace.from_config()
111+
region = ""
112+
subscription_id = ""
113+
resource_group = ""
114+
workspace_name = ""
57115

58-
mlflow.set_tracking_uri(ws.get_mlflow_tracking_uri())
116+
azureml_mlflow_uri = f"azureml://{region}.api.azureml.ms/mlflow/v1.0/subscriptions/{subscription_id}/resourceGroups/{resource_group}/providers/Microsoft.MachineLearningServices/workspaces/{workspace_name}"
117+
mlflow.set_tracking_uri(azureml_mlflow_uri)
59118
```
60119

61-
Set the MLflow experiment name with `set_experiment()` and start your training run with `start_run()`. Then, use `log_metric()` to activate the MLflow logging API and begin logging your training run metrics.
120+
> [!NOTE]
121+
> You can also get this URL by:
122+
> 1. Navigate to [Azure ML studio](https://ml.azure.com)
123+
> 2. Click on the uper-right corner of the page -> View all properties in Azure Portal -> MLflow tracking URI.
124+
> 3. Copy the URI and use it with the method `mlflow.set_tracking_uri`.
62125

63-
```Python
64-
experiment_name = 'experiment-with-mlflow-projects'
65-
mlflow.set_experiment(experiment_name)
66-
```
126+
---
127+
128+
## Train MLflow Projects on local compute
129+
130+
This example shows how to submit MLflow projects locally with Azure Machine Learning.
67131

68132
Create the backend configuration object to store necessary information for the integration such as, the compute target and which type of managed environment to use.
69133

@@ -106,30 +170,6 @@ local_env_run = mlflow.projects.run(uri=".",
106170

107171
This example shows how to submit MLflow projects on a remote compute with Azure Machine Learning tracking.
108172

109-
Install the `azureml-mlflow` package to use MLflow Tracking with Azure Machine Learning on your experiments locally. Your experiments can run via a Jupyter Notebook or code editor.
110-
111-
```shell
112-
pip install azureml-mlflow
113-
```
114-
115-
Import the `mlflow` and [`Workspace`](/python/api/azureml-core/azureml.core.workspace%28class%29) classes to access MLflow's tracking URI and configure your workspace.
116-
117-
```Python
118-
import mlflow
119-
from azureml.core import Workspace
120-
121-
ws = Workspace.from_config()
122-
123-
mlflow.set_tracking_uri(ws.get_mlflow_tracking_uri())
124-
```
125-
126-
Set the MLflow experiment name with `set_experiment()` and start your training run with `start_run()`. Then, use `log_metric()` to activate the MLflow logging API and begin logging your training run metrics.
127-
128-
```Python
129-
experiment_name = 'train-mlflow-project-amlcompute'
130-
mlflow.set_experiment(experiment_name)
131-
```
132-
133173
Create the backend configuration object to store necessary information for the integration such as, the compute target and which type of managed environment to use.
134174

135175
The integration accepts "COMPUTE" and "USE_CONDA" as parameters where "COMPUTE" is set to the name of your remote compute cluster and "USE_CONDA" which creates a new environment for the project from the environment configuration file. If "COMPUTE" is present in the object, the project will be automatically submitted to the remote compute and ignore "USE_CONDA". MLflow accepts a dictionary object or a JSON file.

0 commit comments

Comments
 (0)