Merge pull request #392 from v-thepet/hyper

Freshness 4 - Azure Machine Learning Pipelines and RAI

Author: denrea

articles/machine-learning/how-to-use-sweep-in-pipeline.md

@@ -1,91 +1,103 @@
 ---
-title: How to do hyperparameter sweep in pipeline
+title: How to do hyperparameter sweep in pipelines
 titleSuffix: Azure Machine Learning
-description: How to use sweep to do hyperparameter tuning in Azure Machine Learning pipeline using CLI v2 and Python SDK
+description: Learn how to use sweep to automate hyperparameter tuning in Azure Machine Learning pipelines with CLI v2 and Python SDK v2.
 services: machine-learning
 ms.service: azure-machine-learning
 ms.subservice: mlops
 ms.topic: how-to
 author: lgayhardt
 ms.author: lagayhar
 ms.reviewer: zhanxia
-ms.date: 05/26/2022
+ms.date: 09/19/2024
 ms.custom: devx-track-python, sdkv2, cliv2, update-code2
 ---
 
-# How to do hyperparameter tuning in pipeline (v2)
+# How to do hyperparameter tuning in pipelines
 
 [!INCLUDE [dev v2](includes/machine-learning-dev-v2.md)]
 
-In this article, you'll learn how to do hyperparameter tuning in Azure Machine Learning pipeline.
+In this article, you learn how to automate hyperparameter tuning in Azure Machine Learning pipelines by using Azure Machine Learning CLI v2 or Azure Machine Learning SDK for Python v2.
 
-## Prerequisite
+Hyperparameters are adjustable parameters that let you control the model training process. Hyperparameter tuning is the process of finding the configuration of hyperparameters that results in the best performance. Azure Machine Learning lets you automate hyperparameter tuning and [run experiments in parallel](how-to-use-parallel-job-in-pipeline.md) to efficiently optimize hyperparameters.
 
-1. Understand what is [hyperparameter tuning](how-to-tune-hyperparameters.md) and how to do hyperparameter tuning in Azure Machine Learning use SweepJob.
-2. Understand what is a [Azure Machine Learning pipeline](concept-ml-pipelines.md)
-3. Build a command component that takes hyperparameter as input.
+## Prerequisites
 
-## How to do hyperparameter tuning in Azure Machine Learning pipeline
+- Have an Azure Machine Learning account and workspace.
+- Understand [Azure Machine Learning pipelines](concept-ml-pipelines.md) and [hyperparameter tuning a model](how-to-tune-hyperparameters.md).
 
-This section explains how to do hyperparameter tuning in Azure Machine Learning pipeline using CLI v2 and Python SDK. Both approaches share the same prerequisite: you already have a command component created and the command component takes hyperparameters as inputs. If you don't have a command component yet. Follow below links to create a command component first.
+## Create and run a hyperparameter tuning pipeline
 
-- [Azure Machine Learning CLI v2](how-to-create-component-pipelines-cli.md)
-- [Azure Machine Learning Python SDK v2](how-to-create-component-pipeline-python.md)
+# [Azure CLI](#tab/cli)
 
-### CLI v2
+The following examples come from [Run a pipeline job using sweep (hyperdrive) in pipeline](https://github.com/Azure/azureml-examples/tree/main/cli/jobs/pipelines-with-components/pipeline_with_hyperparameter_sweep) in the [Azure Machine Learning examples](https://github.com/Azure/azureml-examples) repository. For more information about creating pipelines with components, see [Create and run machine learning pipelines using components with the Azure Machine Learning CLI](how-to-create-component-pipelines-cli.md).
 
-The example used in this article can be found in [azureml-example repo](https://github.com/Azure/azureml-examples). Navigate to *[azureml-examples/cli/jobs/pipelines-with-components/pipeline_with_hyperparameter_sweep* to check the example.
+# [Python SDK](#tab/python)
 
-Assume you already have a command component defined in `train.yaml`. A two-step pipeline job (train and predict) YAML file looks like below.
+The following examples come from the [Build pipeline with sweep node](https://github.com/Azure/azureml-examples/blob/main/sdk/python/jobs/pipelines/1c_pipeline_with_hyperparameter_sweep/pipeline_with_hyperparameter_sweep.ipynb) notebook in the [Azure Machine Learning examples](https://github.com/Azure/azureml-examples) repository. For more information about creating pipelines with components, see [Create and run machine learning pipelines using components with the Azure Machine Learning SDK v2](how-to-create-component-pipeline-python.md).
 
-:::code language="yaml" source="~/azureml-examples-main/cli/jobs/pipelines-with-components/pipeline_with_hyperparameter_sweep/pipeline.yml" highlight="7-48":::
+For a related notebook, see [Run hyperparameter sweep on a command job](https://github.com/Azure/azureml-examples/blob/main/sdk/python/jobs/single-step/lightgbm/iris/lightgbm-iris-sweep.ipynb).
 
-The `sweep_step` is the step for hyperparameter tuning. Its type needs to be `sweep`.  And `trial` refers to the command component defined in `train.yaml`. From the `search space` field we can see three hyparmeters (`c_value`, `kernel`, and `coef`) are added to the search space. After you submit this pipeline job, Azure Machine Learning will run the trial component multiple times to sweep over hyperparameters based on the search space and terminate policy you defined in `sweep_step`. Check [sweep job YAML schema](reference-yaml-job-sweep.md) for full schema of sweep job.
+---
+
+### Create a command component with hyperparameter inputs
 
-Below is the trial component definition (train.yml file). 
+The Azure Machine Learning pipeline must have a command component with hyperparameter inputs. The following *train.yml* file from the example projects defines a `trial` component that has the `c_value`, `kernel`, and `coef` hyperparameter inputs and runs the source code that's located in the *./train-src* folder.
 
 :::code language="yaml" source="~/azureml-examples-main/cli/jobs/pipelines-with-components/pipeline_with_hyperparameter_sweep/train.yml" highlight="11-16,23-25,60":::
 
-The hyperparameters added to search space in pipeline.yml need to be inputs for the trial component. The source code of the trial component is under `./train-src` folder. In this example, it's a single `train.py` file. This is the code that will be executed in every trial of the sweep job. Make sure you've logged the metrics in the trial component source code with exactly the same name as `primary_metric` value in pipeline.yml file. In this example, we use `mlflow.autolog()`, which is the recommended way to track your ML experiments. See more about mlflow [here](./how-to-use-mlflow-cli-runs.md)  
- 
-Below code snippet is the source code of trial component. 
+### Create the trial component source code
+
+The source code for this example is a single *train.py* file. This code executes in every trial of the sweep job.
 
 :::code language="python" source="~/azureml-examples-main/cli/jobs/pipelines-with-components/pipeline_with_hyperparameter_sweep/train-src/train.py" highlight="15":::
 
-### Python SDK
+>[!NOTE]
+>Make sure to log the metrics in the trial component source code with exactly the same name as the `primary_metric` value in the pipeline file. This example uses `mlflow.autolog()`, which is the recommended way to track machine learning experiments. For more information about MLflow, see [Track ML experiments and models with MLflow](./how-to-use-mlflow-cli-runs.md).
 
-The Python SDK example can be found in [azureml-example repo](https://github.com/Azure/azureml-examples). Navigate to *azureml-examples/sdk/jobs/pipelines/1c_pipeline_with_hyperparameter_sweep* to check the example.
+### Create a pipeline with a hyperparameter sweep step
 
-In Azure Machine Learning Python SDK v2, you can enable hyperparameter tuning for any command component by calling `.sweep()` method.
+# [Azure CLI](#tab/cli)
 
-Below code snippet shows how to enable sweep for `train_model`.
+Given the command component defined in *train.yml*, the following code creates a two-step `train` and `predict` pipeline definition file. In the `sweep_step`, the required step type is `sweep`, and the `c_value`, `kernel`, and `coef` hyperparameter inputs for the `trial` component are added to the `search_space`.
 
-[!notebook-python[] (~/azureml-examples-main/sdk/python/jobs/pipelines/1c_pipeline_with_hyperparameter_sweep/pipeline_with_hyperparameter_sweep.ipynb?name=enable-sweep)]
+The following example highlights the hyperparameter tuning `sweep_step`.
 
- We first load `train_component_func` defined in `train.yml` file. When creating `train_model`, we add `c_value`, `kernel` and `coef0` into search space(line 15-17). Line 30-35 defines the primary metric, sampling algorithm etc.
+:::code language="yaml" source="~/azureml-examples-main/cli/jobs/pipelines-with-components/pipeline_with_hyperparameter_sweep/pipeline.yml" highlight="8-48":::
 
-## Check pipeline job with sweep step in Studio
+# [Python SDK](#tab/python)
 
-After you submit a pipeline job, the SDK or CLI widget will give you a web URL link to Studio UI. The link will guide you to the pipeline graph view by default.
+In the v2 SDK, you can enable hyperparameter tuning for any command component by calling the `.sweep()` method. The following pipeline definition shows how to enable sweep for `train_model`.
 
-To check details of the sweep step, double click the sweep step and navigate to the **child job** tab in the panel on the right.
+The example first loads the `train_component_func` defined in the *train.yml* file. To create the `train_model`, the code adds the `c_value`, `kernel`, and `coef0` hyperparameters into the search space. The `sweep_step` defines the `primary_metric`, `sampling_algorithm`, and other parameters.
 
-:::image type="content" source="./media/how-to-use-sweep-in-pipeline/pipeline-view.png" alt-text="Screenshot of the pipeline with child job and the train_model node highlighted." lightbox= "./media/how-to-use-sweep-in-pipeline/pipeline-view.png":::
+[!Notebook-python[] (~/azureml-examples-main/sdk/python/jobs/pipelines/1c_pipeline_with_hyperparameter_sweep/pipeline_with_hyperparameter_sweep.ipynb?name=enable-sweep)]
+
+---
 
-This will link you to the sweep job page as seen in the below screenshot. Navigate to **child job** tab, here you can see the metrics of all child jobs and list of all child jobs.
+For the full sweep job schema, see [CLI (v2) sweep job YAML schema](reference-yaml-job-sweep.md).
 
-:::image type="content" source="./media/how-to-use-sweep-in-pipeline/sweep-job.png" alt-text="Screenshot of the job page on the child jobs tab." lightbox= "./media/how-to-use-sweep-in-pipeline/sweep-job.png":::
+### Submit the hyperparameter tuning pipeline job
+
+After you submit this pipeline job, Azure Machine Learning runs the `trial` component multiple times to sweep over hyperparameters, based on the search space and limits you defined in the `sweep_step`.
+
+## View hyperparameter tuning results in studio
+
+After you submit a pipeline job, the SDK or CLI widget gives you a web URL link to the pipeline graph in the Azure Machine Learning studio UI.
+
+To view hyperparameter tuning results, double-click the sweep step in the pipeline graph, select the **Child jobs** tab in the details panel, and then select the child job.
+
+:::image type="content" source="./media/how-to-use-sweep-in-pipeline/pipeline-view.png" alt-text="Screenshot of the pipeline with child job and the train_model node highlighted." lightbox= "./media/how-to-use-sweep-in-pipeline/pipeline-view.png":::
 
-If a child jobs failed, select the name of that child job to enter detail page of that specific child job (see screenshot below). The useful debug information is under **Outputs + Logs**.
+On the child job page, select the **Trials** tab to see and compare metrics for all the child runs. Select any of the child runs to see the details for that run.
 
-:::image type="content" source="./media/how-to-use-sweep-in-pipeline/child-run.png" alt-text="Screenshot of the output + logs tab of a child run." lightbox= "./media/how-to-use-sweep-in-pipeline/child-run.png":::
+:::image type="content" source="./media/how-to-use-sweep-in-pipeline/sweep-job.png" alt-text="Screenshot of the child job page with the Trials tab." lightbox= "./media/how-to-use-sweep-in-pipeline/sweep-job.png":::
 
-## Sample notebooks
+If a child run failed, you can select the **Outputs + logs** tab on the child run page to see useful debug information.
 
-- [Build pipeline with sweep node](https://github.com/Azure/azureml-examples/blob/main/sdk/python/jobs/pipelines/1c_pipeline_with_hyperparameter_sweep/pipeline_with_hyperparameter_sweep.ipynb)
-- [Run hyperparameter sweep on a command job](https://github.com/Azure/azureml-examples/blob/main/sdk/python/jobs/single-step/lightgbm/iris/lightgbm-iris-sweep.ipynb)
+:::image type="content" source="./media/how-to-use-sweep-in-pipeline/child-run.png" alt-text="Screenshot of the output and logs tab of a child run." lightbox= "./media/how-to-use-sweep-in-pipeline/child-run.png":::
 
-## Next steps
+## Related content
 
 - [Track an experiment](how-to-log-view-metrics.md)
 - [Deploy a trained model](how-to-deploy-online-endpoints.md)