You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Parallel job lets users accelerate their job execution by distributing repeated tasks on powerful multi-nodes compute clusters. For example, take the scenario where you are running a object detection model on large set of images. With Azure ML Parallel job, you can easily distribute your images to run custom code in parallel on a specific compute cluster. Parallelization could significantly reduce the time cost. Also by using Azure ML parallel job you can simplify and automate your process to make it more efficient.
19
+
Parallel job lets users accelerate their job execution by distributing repeated tasks on powerful multi-nodes compute clusters. For example, take the scenario where you're running an object detection model on large set of images. With Azure ML Parallel job, you can easily distribute your images to run custom code in parallel on a specific compute cluster. Parallelization could significantly reduce the time cost. Also by using Azure ML parallel job you can simplify and automate your process to make it more efficient.
20
20
21
21
## Prerequisite
22
22
23
23
Azure ML parallel job can only be used as one of steps in a pipeline job. Thus, it's important to be familiar with using pipelines. To learn more about Azure ML pipelines, see the following articles.
24
24
25
-
1. Understand what is a [Azure Machine Learning pipeline](concept-ml-pipelines.md)
26
-
2. Understand how to use Azure ML pipeline with [CLI v2](how-to-create-component-pipelines-cli.md) and [SDK v2](how-to-create-component-pipeline-python.md).
25
+
- Understand what is a [Azure Machine Learning pipeline](concept-ml-pipelines.md)
26
+
- Understand how to use Azure ML pipeline with [CLI v2](how-to-create-component-pipelines-cli.md) and [SDK v2](how-to-create-component-pipeline-python.md).
27
27
28
28
## Why are parallel jobs needed?
29
29
@@ -32,7 +32,7 @@ In the real world, ML engineers always have scale requirements on their training
32
32
- Delay pressure caused by long execution time.
33
33
- Manual intervention to handle unexpected issues to keep the task proceeding.
34
34
35
-
The core value of Azure ML parallel job is to split a single serial task into mini-batches and dispatch those mini-batches to multiple computes to execute in parallel. By using parallel jobs we can:
35
+
The core value of Azure ML parallel job is to split a single serial task into mini-batches and dispatch those mini-batches to multiple computes to execute in parallel. By using parallel jobs, we can:
36
36
37
37
- Significantly reduce end-to-end execution time.
38
38
- Use Azure ML parallel job's automatic error handling settings.
0 commit comments