You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/concept-pipeline-practices-tips.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,9 +26,9 @@ There are several options for getting started if your are new to pipelines:
26
26
27
27
## How do you modularize pipeline code?
28
28
29
-
Modules and the `ModuleStep` class give you a great opportunity to modularize your ML code. However, it has to be kept in mind that moving between pipeline steps is vastly more expensive than a function call. The question you must ask is not so much "Are these functions and data conceptually different than those in this other section?" but "Do I want these functions and data to evolve separately?" or "Is this an expensive computation whose output I can reuse?" For more information, see this notebook[How to create Module, ModuleVersion, and use them in a pipeline with ModuleStep](https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/machine-learning-pipelines/intro-to-pipelines/aml-pipelines-how-to-use-modulestep.ipynb).
29
+
Modules and the `ModuleStep` class give you a great opportunity to modularize your ML code. However, it has to be kept in mind that moving between pipeline steps is vastly more expensive than a function call. The question you must ask isn't so much "Are these functions and data conceptually different than the ones in this other section?" but "Do I want these functions and data to evolve separately?" or "Is this an expensive computation whose output I can reuse?" For more information, see thisn'tebook[How to create Module, ModuleVersion, and use them in a pipeline with ModuleStep](https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/machine-learning-pipelines/intro-to-pipelines/aml-pipelines-how-to-use-modulestep.ipynb).
30
30
31
-
As discussed previously, separating data preparation from training is often one such opportunity. Sometimes data preparation is complex and time-consuming enough that it can be appropriate to break into separate pipeline steps. Other opportunities include post-training testing and analysis.
31
+
As discussed previously, separating data preparation from training is often one such opportunity. Sometimes data preparation is complex and time-consuming enough that you might break the process into separate pipeline steps. Other opportunities include post-training testing and analysis.
32
32
33
33
## How do you speed pipeline iteration?
34
34
@@ -38,19 +38,19 @@ Common techniques for quickly iterating pipelines include:
38
38
- Keeping the compute instance running, so as to avoid startup time
39
39
- Configuring data and steps to allow reuse will allow the pipeline to skip recalculating unchanging data
40
40
41
-
When you want to quickly iterate, you can clone your pipeline, making a pipeline, and rerun the pipeline. Another helpful technique is If you keep your Compute warm, you will not incur the cost of spinning up the new compute. If you set up the Step to allow reuse of the result of a run, then the repeated execution will reuse results where possible (when there are no change in the Steps).
41
+
When you want to quickly iterate, you can clone your pipeline, making a pipeline, and rerun the pipeline. Another helpful technique is If you keep your Compute warm, you won't incur the cost of spinning up the new compute. If you set up the Step to allow reuse of the result of a run, then the repeated execution will reuse results where possible (when there are no change in the Steps).
42
42
43
43
## How do you collaborate using ML pipelines?
44
44
45
45
Separate pipelines are natural lines along which to split effort. Multiple developers or even multiple teams can work on different steps, so long as the data and arguments flowing between steps are agreed upon.
46
46
47
-
During active development, you can retrieve `PipelineRun` and `StepRun`metadata from the workspace and use these to download final and intermediate output and artifacts, and use those for your own modularized work.
47
+
During active development, you can retrieve `PipelineRun` and `StepRun`run results from the workspace, use these objects to download final and intermediate output and artifacts, and use those for your own modularized work.
48
48
49
49
## Use pipelines to test techniques in isolation
50
50
51
-
Real-world ML solutions generally involve considerable customization of every step. The raw data often needs to be filtered, transformed, and augmented. The training processes might have several potential architectures and, for deep learning, many possible variations in terms of layer sizes and activation functions. Even with a consistent architecture, hyperparameter search can produce significant wins.
51
+
Real-world ML solutions generally involve considerable customization of every step. The raw data often needs to be filtered, transformed, and augmented. The training processes might have several potential architectures and, for deep learning, many possible variations for layer sizes and activation functions. Even with a consistent architecture, hyperparameter search can produce significant wins.
52
52
53
-
In addition to tools like [AutoML](concept-automated-ml.md) and [automated hyperparameter search](how-to-tune-hyperparameters.md), pipelines can be an important tool for A/B testing solutions. If you have several variants of your pipeline steps, it is easy to generate separate runs trying their variations:
53
+
In addition to tools like [AutoML](concept-automated-ml.md) and [automated hyperparameter search](how-to-tune-hyperparameters.md), pipelines can be an important tool for A/B testing solutions. If you have several variants of your pipeline steps, it's easy to generate separate runs trying their variations:
0 commit comments