first pass

Author: v-thepet

articles/batch/batch-job-task-error-checking.md

@@ -1,88 +1,99 @@
 ---
 title: Check for job and task errors
-description: Learn about errors to check for and how to troubleshoot jobs and tasks.
+description: Learn how to check for and handle errors that occur after Azure Batch jobs and tasks are submitted..
 ms.topic: how-to
-ms.date: 09/08/2021
+ms.date: 04/11/2023
 ---
 
-# Job and task error checking
+# Azure Batch job and task errors
 
-There are various errors that can occur when adding jobs and tasks. Detecting failures for these operations is straightforward because any failures are returned immediately by the API, CLI, or UI. However, there are also failures that can happen later, when jobs and tasks are scheduled and run.
+Various errors can happen when you add Azure Batch jobs and tasks or when you schedule and run jobs and tasks. It's straightforward to detect errors that occur when you add jobs and tasks, because any failures are returned immediately by the API, command line, or user interface. This article covers how to check for and handle errors that occur after jobs and tasks are submitted.
 
-This article covers the errors that can occur after jobs and tasks are submitted and how to check for and handle them.
+## Job failures
 
-## Jobs
+A job is a group of one or more tasks, which specify command lines to run. You can specify the following optional parameters when you add a job, which influence how the job can fail:
 
-A job is a grouping of one or more tasks, with the tasks actually specifying the command lines to be run.
+- [JobConstraints](/rest/api/batchservice/job/add#jobconstraints). You can optionally use the `maxWallClockTime` property to set the maximum amount of time a job can be active or running. If the job exceeds the `maxWallClockTime`, the job terminates with the `terminateReason` property set in the [JobExecutionInformation](/rest/api/batchservice/job/get#jobexecutioninformation).
 
-When adding a job, the following parameters can be specified which can influence how the job can fail:
+- [JobPreparationTask](/rest/api/batchservice/job/add#jobpreparationtask). You can optionally specify a job preparation task to run on a node the first time it runs a task for the job. If the job preparation task fails, the task doesn't run and the job doesn't complete.
 
-- [Job Constraints](/rest/api/batchservice/job/add#jobconstraints)
-  - The `maxWallClockTime` property can optionally be specified to set the maximum amount of time a job can be active or running. If exceeded, the job will be terminated with the `terminateReason` property set in the [executionInfo](/rest/api/batchservice/job/get#jobexecutioninformation) for the job.
-- [Job Preparation Task](/rest/api/batchservice/job/add#jobpreparationtask)
-  - If specified, a job preparation task is run the first time a task is run for a job on a node. The job preparation task can fail, which will lead to the task not being run and the job not completing.
-- [Job Release Task](/rest/api/batchservice/job/add#jobreleasetask)
-  - A job release task can only be specified if a job preparation task is configured. When a job is being terminated, the job release task is run on the each of pool nodes where a job preparation task was run. A job release task can fail, but the job will still move to a `completed` state.
+- [JobReleaseTask](/rest/api/batchservice/job/add#jobreleasetask). You can optionally specify a job release task for jobs that have a job preparation task. When a job is being terminated, the job release task runs on each pool node that ran a job preparation task. If a job release task fails, the job still moves to a `completed` state.
+
+In the Azure portal, you can set these parameters in the **Job manager, preparation and release tasks** and **Advanced** sections of the Batch **Add job** screen.
 
 ### Job properties
 
-The following job properties should be checked for errors:
+Check the following job properties in the [JobExecutionInformation](/rest/api/batchservice/job/get#jobexecutioninformation) for errors:
+
+- The `terminateReason` property indicates `MaxWallClockTimeExpiry` if the job exceeded the `maxWallClockTime` specified in the job constraints and therefore the job terminated. This property can also be set to `taskFailed` if the job's `onTaskFailure` attribute is set to `performExitOptionsJobAction`, and a task fails with an exit condition that specifies a `jobAction` of `terminatejob`. 
 
-- '[executionInfo](/rest/api/batchservice/job/get#jobexecutioninformation)':
-  - The `terminateReason` property can have values to indicate that the `maxWallClockTime`, specified in the job constraints, was exceeded and therefore the job was terminated. It can also be set to indicate a task failed if the job `onTaskFailure` property was set appropriately.
-  - The [schedulingError](/rest/api/batchservice/job/get#jobschedulingerror) property is set if there has been a scheduling error.
+- The [JobSchedulingError](/rest/api/batchservice/job/get#jobschedulingerror) property is set if there has been a scheduling error.
 
 ### Job preparation tasks
 
-If a [job preparation task](batch-job-prep-release.md#job-preparation-task) is specified for a job, then an instance of that task will be run the first time a task for the job is run on a node. The job preparation task configured on the job can be thought of as a task template, with multiple job preparation task instances being run, up to the number of nodes in a pool.
+If you specify a [job preparation task](batch-job-prep-release.md#job-preparation-task) for a job, an instance of the job preparation task runs on a node the first time the node runs a task for that job. You can think of the job preparation task as a task template, with multiple instances being run, up to the number of nodes in a pool. Check the job preparation task instances to determine if there were errors.
+
+You can use the [Job - List Preparation and Release Task Status](/rest/api/batchservice/job/listpreparationandreleasetaskstatus) API to list the execution status of all instances of job preparation and release tasks for a specified job. As with other tasks, [JobPreparationTaskExecutionInformation](/rest/api/batchservice/job/listpreparationandreleasetaskstatus#jobpreparationtaskexecutioninformation) is available with properties such as `failureInfo`, `exitCode`, and `result`.
 
-The job preparation task instances should be checked to determine if there were errors:
+When a job preparation task runs, the task that triggered the job preparation task moves to a [taskState](/rest/api/batchservice/task/get#taskstate) of `preparing`. If the job preparation task fails, the triggering task reverts to the `active` state and doesn't run.
 
-- When a job preparation task is run, then the task that triggered the job preparation task will move to a [state](/rest/api/batchservice/task/get#taskstate) of `preparing`; if the job preparation task then fails, the triggering task will revert to the `active` state and will not be run.
-- All the instances of the job preparation task that have been run can be obtained from the job using the [List Preparation and Release Task Status](/rest/api/batchservice/job/listpreparationandreleasetaskstatus) API. As with any task, there is [execution information](/rest/api/batchservice/job/listpreparationandreleasetaskstatus#jobpreparationandreleasetaskexecutioninformation) available with properties such as `failureInfo`, `exitCode`, and `result`.
-- If job preparation tasks fail, then the triggering job tasks will not be run, the job will not complete and will be stuck. The pool may go unutilized if there are no other jobs with tasks that can be scheduled.
+If job preparation tasks fail and the triggering job task doesn't run, the job doesn't complete and is stuck. The pool might not be used if there are no other jobs with tasks that can be scheduled.
 
 ### Job release tasks
 
-If a [job release task](batch-job-prep-release.md#job-release-task) is specified for a job, then when a job is being terminated, an instance of the job release task is run on each pool node where a job preparation task was run. The job release task instances should be checked to determine if there were errors:
+If you specify a [job release task](batch-job-prep-release.md#job-release-task) for a job, an instance of the job release task runs on each pool node where a job preparation task ran when the job is being terminated. Check the job release task instances to determine if there were errors.
+
+You can use the [Job - List Preparation and Release Task Status](/rest/api/batchservice/job/listpreparationandreleasetaskstatus) API to list the execution status of all instances of job preparation and release tasks for a specified job. As with other tasks, [JobReleaseTaskExecutionInformation](/rest/api/batchservice/job/listpreparationandreleasetaskstatus#jobreleasetaskexecutioninformation) is available with properties such as `failureInfo`, `exitCode`, and `result`.
+
+If one or more job release tasks fail, the job is still terminated and moves to a `completed` state.
+
+## Task failures
+
+Job tasks can fail for the following reasons:
+
+- The task command line fails and returns with a non-zero exit code.
+- One or more `resourceFiles` specified for a task don't download.
+- One or more `outputFiles` specified for a task don't upload.
+- The elapsed time for the task exceeds the `maxWallClockTime` property specified in the [TaskConstraints](/rest/api/batchservice/task/add#taskconstraints).
+
+In all cases, check the following properties for errors and information about the errors:
+
+- The [TaskExecutionInformation](/rest/api/batchservice/task/get#taskexecutioninformation) property has multiple properties that provide information about an error. The [taskExecutionResult](/rest/api/batchservice/task/get#taskexecutionresult) indicates if the task failed for any reason, and `exitCode` and `failureInfo` provide more information about the failure.
+
+- The task always moves to the `completed` [TaskState](/rest/api/batchservice/task/get#taskstate), whether it succeeded or failed.
 
-- All the instances of the job release task being run can be obtained from the job using the API [List Preparation and Release Task Status](/rest/api/batchservice/job/listpreparationandreleasetaskstatus). As with any task, there is [execution information](/rest/api/batchservice/job/listpreparationandreleasetaskstatus#jobpreparationandreleasetaskexecutioninformation) available with properties such as `failureInfo`, `exitCode`, and `result`.
-- If one or more job release tasks fail, then the job will still be terminated and move to a `completed` state.
+Consider the impact of task failures on the job and on any task dependencies. You can specify [ExitConditions](/rest/api/batchservice/task/add#exitconditions) to configure actions for dependencies and for the job.
 
-## Tasks
+- [DependencyAction](/rest/api/batchservice/task/add#dependencyaction) controls whether to block or run tasks that depend on the failed task.
+- [JobAction](/rest/api/batchservice/task/add#jobaction) controls whether the failed task causes the job to be disabled, terminated, or unchanged.
 
-Job tasks can fail for multiple reasons:
+### Task command lines, output files, and logs
 
-- The task command line fails, returning with a non-zero exit code.
-- There are `resourceFiles` specified for a task, but there was a failure that meant one or more files didn't download.
-- There are `outputFiles` specified for a task, but there was a failure that meant one or more files didn't upload.
-- The elapsed time for the task, specified by the `maxWallClockTime` property in the task [constraints](/rest/api/batchservice/task/add#taskconstraints), was exceeded.
+Task command lines don't run under a shell on compute nodes, so they can't natively use shell features such as environment variable expansion. To take advantage of such features, you must invoke the shell in the command line. For more information, see [Command-line expansion of environment variables](batch-compute-node-environment-variables.md#command-line-expansion-of-environment-variables).
 
-In all cases the following properties must be checked for errors and information about the errors:
+Task command line output writes to *stderr.txt* and *stdout.txt*. Your application might also write to application-specific log files. Make sure to implement comprehensive error checking for your application to promptly detect and diagnose issues.
 
-- The tasks [executionInfo](/rest/api/batchservice/task/get#taskexecutioninformation) property contains multiple properties that provide information about an error. [result](/rest/api/batchservice/task/get#taskexecutionresult) indicates if the task failed for any reason, with `exitCode` and `failureInfo` providing more information about the failure.
-- The task will always move to the `completed` [state](/rest/api/batchservice/task/get#taskstate), independent of whether it succeeded or failed.
+If the pool node that ran a task still exists, you can get and view the log files. Several APIs allow listing and getting task files, such as [File - Get From Task](/rest/api/batchservice/file/getfromtask). You can also list and view log files for a task or pool node by using the Azure portal.
 
-The impact of task failures on the job and any task dependencies must be considered. The [exitConditions](/rest/api/batchservice/task/add#exitconditions) property can be specified for a task to configure an action for dependencies and for the job.
+1. At the top of the **Overview** page for a node, select **Upload batch logs**.
 
-- For dependencies, [DependencyAction](/rest/api/batchservice/task/add#dependencyaction) controls whether the tasks dependent on the failed task are blocked or are run.
-- For the job, [JobAction](/rest/api/batchservice/task/add#jobaction) controls whether the failed task leads to the job being disabled, terminated, or left unchanged.
+   ![Screenshot of a node overview page with Upload batch logs highlighted.](media/node-page.png)
 
-### Task command line failures
+1. On the **Upload Batch logs** page, select **Pick storage container**, select an Azure Storage container to upload to, and then select **Start upload**.
 
-When the task command line is run, output is written to `stderr.txt` and `stdout.txt`. In addition, the application may write to application-specific log files.
+   ![Screenshot of the Upload batch logs page.](media/upload-batch-logs.png)
 
-If the pool node on which a task has run still exists, then the log files can be obtained and viewed. For example, the Azure portal lists and can view log files for a task or a pool node. Multiple APIs also allow task files to be listed and obtained, such as [Get From Task](/rest/api/batchservice/file/getfromtask).
+1. On the storage container page, you can view, open, or download the logs.
 
-Since pools and pool nodes are frequently ephemeral, with nodes being continuously added and deleted, we recommend saving log files. [Task output files](./batch-task-output-files.md) are a convenient way to save log files to Azure Storage.
+   ![Screenshot of task logs in a storage container.](media/task-logs.png)
 
-The command lines executed by tasks on compute nodes do not run under a shell, so they can't natively take advantage of shell features such as environment variable expansion. To take advantage of such features, you must [invoke the shell in the command line](batch-compute-node-environment-variables.md#command-line-expansion-of-environment-variables).
+Because Batch pools and pool nodes are often ephemeral, with nodes being continuously added and deleted, it's best to save the log files from job runs. You can use task output files for a convenient way to save log files to Azure Storage. For more information, see [Persist task data to Azure Storage with the Batch service API](batch-task-output-files.md).
 
 ### Output file failures
 
-On every file upload, Batch writes two log files to the compute node, `fileuploadout.txt` and `fileuploaderr.txt`. You can examine these log files to learn more about a specific failure. In cases where the file upload was never attempted, for example because the task itself couldn't run, then these log files will not exist.  
+On every file upload, Batch writes two log files to the compute node, *fileuploadout.txt* and *fileuploaderr.txt*. You can examine these log files to learn more about a specific failure. If the file upload wasn't attempted, for example because the task itself couldn't run, these log files don't exist.
 
 ## Next steps
 
-- Check that your application implements comprehensive error checking; it can be critical to promptly detect and diagnose issues.
-- Learn more about [jobs and tasks](jobs-and-tasks.md) and [job preparation and release tasks](batch-job-prep-release.md).
+- Learn more about [Batch jobs and tasks](jobs-and-tasks.md) and [job preparation and release tasks](batch-job-prep-release.md).
+- Learn about [Batch pool and node errors](batch-pool-node-error-checking.md).