Skip to content

Commit f93783c

Browse files
author
Larry O'Brien
committed
Added multistep pipeline example
1 parent 5d85eed commit f93783c

File tree

1 file changed

+59
-1
lines changed

1 file changed

+59
-1
lines changed

articles/machine-learning/reference-pipeline-yaml.md

Lines changed: 59 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -315,7 +315,6 @@ pipeline:
315315

316316
| YAML key | Description |
317317
| ----- | ----- |
318-
| `compute_target` | The compute target to use for this step. The compute target can be an Azure Machine Learning Compute, Virtual Machine (such as the Data Science VM), or HDInsight. |
319318
| `inputs` | Inputs can be [InputPortBinding](https://docs.microsoft.com/python/api/azureml-pipeline-core/azureml.pipeline.core.graph.inputportbinding?view=azure-ml-py), [DataReference](#data-reference), [PortDataReference](https://docs.microsoft.com/python/api/azureml-pipeline-core/azureml.pipeline.core.portdatareference?view=azure-ml-py), [PipelineData](https://docs.microsoft.com/python/api/azureml-pipeline-core/azureml.pipeline.core.pipelinedata?view=azure-ml-py), [Dataset](https://docs.microsoft.com/python/api/azureml-core/azureml.core.dataset%28class%29?view=azure-ml-py), [DatasetDefinition](https://docs.microsoft.com/python/api/azureml-core/azureml.data.dataset_definition.datasetdefinition?view=azure-ml-py), or [PipelineDataset](https://docs.microsoft.com/python/api/azureml-pipeline-core/azureml.pipeline.core.pipelinedataset?view=azure-ml-py). |
320319
| `outputs` | Outputs can be either [PipelineData](https://docs.microsoft.com/python/api/azureml-pipeline-core/azureml.pipeline.core.pipelinedata?view=azure-ml-py) or [OutputPortBinding](https://docs.microsoft.com/python/api/azureml-pipeline-core/azureml.pipeline.core.graph.outputportbinding?view=azure-ml-py). |
321320
| `script_name` | The name of the Python script (relative to `source_directory`). |
@@ -359,6 +358,65 @@ pipeline:
359358
bind_mode: mount
360359
```
361360

361+
### Pipeline with multiple steps
362+
363+
| YAML key | Description |
364+
| ----- | ----- |
365+
| `steps` | Sequence of one or more PipelineStep definitions. Note that the `destination` of one step's `outputs` become the keys to the `inputs` of the .|
366+
367+
```yaml
368+
pipeline:
369+
name: SamplePipelineFromYAML
370+
description: Sample multistep YAML pipeline
371+
data_references:
372+
TitanicDS:
373+
dataset_name: 'titanic_ds'
374+
bind_mode: download
375+
default_compute: cpu-cluster
376+
steps:
377+
Dataprep:
378+
type: "PythonScriptStep"
379+
name: "DataPrep Step"
380+
compute: cpu-cluster
381+
runconfig: ".\\default_runconfig.yml"
382+
script_name: "prep.py"
383+
arguments:
384+
- '--train_path'
385+
- output:train_path
386+
- '--test_path'
387+
- output:test_path
388+
allow_reuse: True
389+
inputs:
390+
titanic_ds:
391+
source: TitanicDS
392+
bind_mode: download
393+
outputs:
394+
train_path:
395+
destination: train_csv
396+
datastore: workspaceblobstore
397+
test_path:
398+
destination: test_csv
399+
Training:
400+
type: "PythonScriptStep"
401+
name: "Training Step"
402+
compute: cpu-cluster
403+
runconfig: ".\\default_runconfig.yml"
404+
script_name: "train.py"
405+
arguments:
406+
- "--train_path"
407+
- input:train_path
408+
- "--test_path"
409+
- input:test_path
410+
inputs:
411+
train_path:
412+
source: train_csv
413+
bind_mode: download
414+
test_path:
415+
source: test_csv
416+
bind_mode: download
417+
418+
```
419+
362420
## Schedules
363421

364422
When defining the schedule for a pipeline, it can be either datastore-triggered or recurring based on a time interval. The following are the keys used to define a schedule:

0 commit comments

Comments
 (0)