You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-use-parallel-job-in-pipeline.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -46,7 +46,7 @@ The core job of Azure Machine Learning parallelization is to split a single seri
46
46
47
47
An Azure Machine Learning parallel job can be used only as a step in a pipeline job.
48
48
49
-
# [Azure CLI](#tab/cli)
49
+
# [Azure CLI](#tab/cliv2)
50
50
51
51
The following examples come from [Run a pipeline job using parallel job in pipeline](https://github.com/Azure/azureml-examples/tree/main/cli/jobs/pipelines/iris-batch-prediction-using-parallel/) in the [Azure Machine Learning examples](https://github.com/Azure/azureml-examples) repository.
52
52
@@ -123,7 +123,7 @@ The parallel job executes the functions in each processor, as shown in the follo
123
123
124
124
:::image type="content" source="./media/how-to-use-parallel-job-in-pipeline/how-entry-script-works-in-parallel-job.png" alt-text="Diagram showing how entry script works in parallel job." border="false":::
125
125
126
-
For more details, see the following entry script examples:
126
+
See the following entry script examples:
127
127
128
128
-[Image identification for a list of image files](https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/machine-learning-pipelines/parallel-run/Code/digit_identification.py)
129
129
-[Iris classification for a tabular iris data](https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/machine-learning-pipelines/parallel-run/Code/iris_score.py)
@@ -158,12 +158,12 @@ Azure Machine Learning parallel job exposes many settings that can automatically
158
158
|--|--|--|--|--|--|--|
159
159
|`mini_batch_error_threshold`| integer | Number of failed mini-batches to ignore in this parallel job. If the count of failed mini-batches is higher than this threshold, the parallel job is marked as failed.<br><br>The mini-batch is marked as failed if:<br>- The count of return from `run()` is less than the mini-batch input count.<br>- Exceptions are caught in custom `run()` code.<br><br>`-1` is the default, meaning to ignore all failed mini-batches. |[-1, int.max]|`-1`|`mini_batch_error_threshold`| N/A |
160
160
|`mini_batch_max_retries`| integer | Number of retries when the mini-batch fails or times out. If all retries fail, the mini-batch is marked as failed per the `mini_batch_error_threshold` calculation. |`[0, int.max]`|`2`|`retry_settings.max_retries`| N/A |
161
-
|`mini_batch_timeout`| integer | Timeout in seconds for executing the custom `run()` function. If execution time is higher than this threshold, the mini-batch is aborted and marked as failed to trigger retry. |`(0, 259200]`|`60`|`retry_settings.timeout`| N/A |
161
+
|`mini_batch_timeout`| integer | Time-out in seconds for executing the custom `run()` function. If execution time is higher than this threshold, the mini-batch is aborted and marked as failed to trigger retry. |`(0, 259200]`|`60`|`retry_settings.timeout`| N/A |
162
162
|`item_error_threshold`| integer | The threshold of failed items. Failed items are counted by the number gap between inputs and returns from each mini-batch. If the sum of failed items is higher than this threshold, the parallel job is marked as failed.<br><br>Note: `-1` is the default, meaning to ignore all failures during parallel job. |`[-1, int.max]`|`-1`| N/A |`--error_threshold`|
163
163
|`allowed_failed_percent`| integer | Similar to `mini_batch_error_threshold`, but uses the percent of failed mini-batches instead of the count. |`[0, 100]`|`100`| N/A |`--allowed_failed_percent`|
164
-
|`overhead_timeout`| integer | Timeout in seconds for initialization of each mini-batch. For example, load mini-batch data and pass it to the `run()` function. |`(0, 259200]`|`600`| N/A |`--task_overhead_timeout`|
165
-
|`progress_update_timeout`| integer | Timeout in seconds for monitoring the progress of mini-batch execution. If no progress updates are received within this timeout setting, the parallel job is marked as failed. |`(0, 259200]`| Dynamically calculated by other settings. | N/A |`--progress_update_timeout`|
166
-
|`first_task_creation_timeout`| integer | Timeout in seconds for monitoring the time between the job start and the run of the first mini-batch. |`(0, 259200]`|`600`| N/A | --first_task_creation_timeout |
164
+
|`overhead_timeout`| integer | Time-out in seconds for initialization of each mini-batch. For example, load mini-batch data and pass it to the `run()` function. |`(0, 259200]`|`600`| N/A |`--task_overhead_timeout`|
165
+
|`progress_update_timeout`| integer | Time-out in seconds for monitoring the progress of mini-batch execution. If no progress updates are received within this timeout setting, the parallel job is marked as failed. |`(0, 259200]`| Dynamically calculated by other settings. | N/A |`--progress_update_timeout`|
166
+
|`first_task_creation_timeout`| integer | Time-out in seconds for monitoring the time between the job start and the run of the first mini-batch. |`(0, 259200]`|`600`| N/A | --first_task_creation_timeout |
167
167
|`logging_level`| string | The level of logs to dump to user log files. |`INFO`, `WARNING`, or `DEBUG`|`INFO`|`logging_level`| N/A |
168
168
|`append_row_to`| string | Aggregate all returns from each run of the mini-batch and output it into this file. May refer to one of the outputs of the parallel job by using the expression `${{outputs.<output_name>}}`|||`task.append_row_to`| N/A |
169
169
|`copy_logs_to_parent`| string | Boolean option whether to copy the job progress, overview, and logs to the parent pipeline job. |`True` or `False`|`False`| N/A |`--copy_logs_to_parent`|
0 commit comments