Skip to content

Commit ade4272

Browse files
authored
Merge pull request #95439 from sdgilley/sdg-master
freshness review
2 parents 0fce617 + 9c239a8 commit ade4272

File tree

5 files changed

+32
-15
lines changed

5 files changed

+32
-15
lines changed

articles/machine-learning/service/how-to-create-your-first-pipeline.md

Lines changed: 29 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -9,15 +9,15 @@ ms.topic: conceptual
99
ms.reviewer: sgilley
1010
ms.author: sanpil
1111
author: sanpil
12-
ms.date: 08/09/2019
12+
ms.date: 11/12/2019
1313
ms.custom: seodec18
1414

1515
---
1616

1717
# Create and run machine learning pipelines with Azure Machine Learning SDK
1818
[!INCLUDE [applies-to-skus](../../../includes/aml-applies-to-basic-enterprise-sku.md)]
1919

20-
In this article, you learn how to create, publish, run, and track a [machine learning pipeline](concept-ml-pipelines.md) by using the [Azure Machine Learning SDK](https://docs.microsoft.com/python/api/overview/azure/ml/intro?view=azure-ml-py). Use **ML pipelines** to create a workflow that stitches together various ML phases, and then publish that pipeline into your Azure Machine Learning workspace to access later or share with others. ML pipelines are ideal for batch scoring scenarios, using various computes, reusing steps instead of rerunning them, as well as sharing ML workflows with others.
20+
In this article, you learn how to create, publish, run, and track a [machine learning pipeline](concept-ml-pipelines.md) by using the [Azure Machine Learning SDK](https://docs.microsoft.com/python/api/overview/azure/ml/intro?view=azure-ml-py). Use **ML pipelines** to create a workflow that stitches together various ML phases, and then publish that pipeline into your Azure Machine Learning workspace to access later or share with others. ML pipelines are ideal for batch scoring scenarios, using various computes, reusing steps instead of rerunning them, as well as sharing ML workflows with others.
2121

2222
While you can use a different kind of pipeline called an [Azure Pipeline](https://docs.microsoft.com/azure/devops/pipelines/targets/azure-machine-learning?context=azure%2Fmachine-learning%2Fservice%2Fcontext%2Fml-context&view=azure-devops&tabs=yaml) for CI/CD automation of ML tasks, that type of pipeline is never stored inside your workspace. [Compare these different pipelines](concept-ml-pipelines.md#which-azure-pipeline-technology-should-i-use).
2323

@@ -97,7 +97,7 @@ blob_input_data = DataReference(
9797
path_on_datastore="20newsgroups/20news.pkl")
9898
```
9999

100-
Intermediate data (or output of a step) is represented by a [PipelineData](https://docs.microsoft.com/python/api/azureml-pipeline-core/azureml.pipeline.core.pipelinedata?view=azure-ml-py) object. `output_data1` is produced as the output of a step, and used as the input of one or more future steps. `PipelineData` introduces a data dependency between steps, and creates an implicit execution order in the pipeline.
100+
Intermediate data (or output of a step) is represented by a [PipelineData](https://docs.microsoft.com/python/api/azureml-pipeline-core/azureml.pipeline.core.pipelinedata?view=azure-ml-py) object. `output_data1` is produced as the output of a step, and used as the input of one or more future steps. `PipelineData` introduces a data dependency between steps, and creates an implicit execution order in the pipeline. This object will be used later when creating pipeline steps.
101101

102102
```python
103103
from azureml.pipeline.core import PipelineData
@@ -110,7 +110,7 @@ output_data1 = PipelineData(
110110

111111
## Set up compute target
112112

113-
In Azure Machine Learning, the term computes__ (or __compute target__) refers to the machines or clusters that perform the computational steps in your machine learning pipeline. See [compute targets for model training](how-to-set-up-training-targets.md) for a full list of compute targets and how to create and attach them to your workspace. The process for creating and or attaching a compute target is the same regardless of whether you are training a model or running a pipeline step. After you create and attach your compute target, use the `ComputeTarget` object in your [pipeline step](#steps).
113+
In Azure Machine Learning, the term __computes__ (or __compute target__) refers to the machines or clusters that perform the computational steps in your machine learning pipeline. See [compute targets for model training](how-to-set-up-training-targets.md) for a full list of compute targets and how to create and attach them to your workspace. The process for creating and or attaching a compute target is the same regardless of whether you are training a model or running a pipeline step. After you create and attach your compute target, use the `ComputeTarget` object in your [pipeline step](#steps).
114114

115115
> [!IMPORTANT]
116116
> Performing management operations on compute targets is not supported from inside remote jobs. Since machine learning pipelines are submitted as a remote job, do not use management operations on compute targets from inside the pipeline.
@@ -347,7 +347,18 @@ When you first run a pipeline, Azure Machine Learning:
347347

348348
For more information, see the [Experiment class](https://docs.microsoft.com/python/api/azureml-core/azureml.core.experiment.experiment?view=azure-ml-py) reference.
349349

350+
### View results of a pipeline
350351

352+
See the list of all your pipelines and their run details in the studio:
353+
354+
1. Sign in to [Azure Machine Learning studio](https://ml.azure.com).
355+
356+
1. [View your workspace](how-to-manage-workspace.md#view).
357+
358+
1. On the left, select **Pipelines** to see all your pipeline runs.
359+
![list of machine learning pipelines](./media/how-to-create-your-first-pipeline/pipelines.png)
360+
361+
1. Select a specific pipeline to see the run results.
351362

352363
## GitHub tracking and integration
353364

@@ -404,21 +415,26 @@ response = requests.post(published_pipeline1.endpoint,
404415
"ParameterAssignments": {"pipeline_arg": 20}})
405416
```
406417

407-
### View results of a published pipeline
408418

409-
See the list of all your published pipelines and their run details:
410-
1. Sign in to [Azure Machine Learning](https://ml.azure.com).
419+
### Use published pipelines in the studio
411420

412-
1. [View your workspace](how-to-manage-workspace.md#view) to find the list of pipelines.
413-
![list of machine learning pipelines](./media/how-to-create-your-first-pipeline/list_of_pipelines.png)
414-
415-
1. Select a specific pipeline to see the run results.
421+
You can also run a published pipeline from the studio:
422+
423+
1. Sign in to [Azure Machine Learning studio](https://ml.azure.com).
424+
425+
1. [View your workspace](how-to-manage-workspace.md#view).
426+
427+
1. On the left, select **Endpoints**.
428+
429+
1. On the top, select **Pipeline endpoints**.
430+
![list of machine learning published pipelines](./media/how-to-create-your-first-pipeline/pipeline-endpoints.png)
431+
432+
1. Select a specific pipeline to run, consume, or review results of previous runs of the pipeline endpoint.
416433

417-
These results are also available in your workspace in [Azure Machine Learning](https://ml.azure.com).
418434

419435
### Disable a published pipeline
420436

421-
To hide a pipeline from your list of published pipelines, you disable it:
437+
To hide a pipeline from your list of published pipelines, you disable it, either in the studio or from the SDK:
422438

423439
```
424440
# Get the pipeline by using its ID from Azure Machine Learning studio

articles/machine-learning/service/how-to-train-ml-models.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ ms.custom: seodec18
1717
# Train models with Azure Machine Learning using estimator
1818
[!INCLUDE [applies-to-skus](../../../includes/aml-applies-to-basic-enterprise-sku.md)]
1919

20-
With Azure Machine Learning, you can easily submit your training script to [various compute targets](how-to-set-up-training-targets.md#compute-targets-for-training), using [RunConfiguration object](how-to-set-up-training-targets.md#whats-a-run-configuration) and [ScriptRunConfig object](how-to-set-up-training-targets.md#submit). That pattern gives you a lot of flexibility and maximum control.
20+
With Azure Machine Learning, you can easily submit your training script to [various compute targets](how-to-set-up-training-targets.md#compute-targets-for-training), using a [RunConfiguration object](how-to-set-up-training-targets.md#whats-a-run-configuration) and a [ScriptRunConfig object](how-to-set-up-training-targets.md#submit). That pattern gives you a lot of flexibility and maximum control.
2121

2222
To facilitate deep learning model training, the Azure Machine Learning Python SDK provides an alternative higher-level abstraction, the estimator class, which allows users to easily construct run configurations. You can create and use a generic [Estimator](https://docs.microsoft.com/python/api/azureml-train-core/azureml.train.estimator?view=azure-ml-py) to submit training script using any learning framework you choose (such as scikit-learn) on any compute target you choose, whether it's your local machine, a single VM in Azure, or a GPU cluster in Azure. For PyTorch, TensorFlow and Chainer tasks, Azure Machine Learning also provides respective [PyTorch](https://docs.microsoft.com/python/api/azureml-train-core/azureml.train.dnn.pytorch?view=azure-ml-py), [TensorFlow](https://docs.microsoft.com/python/api/azureml-train-core/azureml.train.dnn.tensorflow?view=azure-ml-py), and [Chainer](https://docs.microsoft.com/python/api/azureml-train-core/azureml.train.dnn.chainer?view=azure-ml-py) estimators to simplify using these frameworks.
2323

@@ -40,7 +40,8 @@ Use an `Estimator` for a single-node training run on remote compute in Azure for
4040
from azureml.train.estimator import Estimator
4141

4242
script_params = {
43-
'--data-folder': ds.as_mount(),
43+
# to mount files referenced by mnist dataset
44+
'--data-folder': ds.as_named_input('mnist').as_mount(),
4445
'--regularization': 0.8
4546
}
4647

58.5 KB
Loading
56.2 KB
Loading

0 commit comments

Comments
 (0)