Merge pull request #215976 from santiagxf/santiagxf/aml-batch-general

AzureML Batch Inference landing page

Author: LJSpiller

articles/machine-learning/.openpublishing.redirection.machine-learning.json

@@ -3499,6 +3499,26 @@
         "source_path_from_root": "/articles/machine-learning/how-to-use-private-python-packages.md",
         "redirect_url": "/azure/machine-learning/v1/how-to-use-private-python-packages",
         "redirect_document_id": true
+    },
+    {
+        "source_path_from_root": "/articles/machine-learning/how-to-use-batch-endpoint.md",
+        "redirect_url": "/azure/machine-learning/batch-inference/how-to-use-batch-endpoint",
+        "redirect_document_id": true
+    },
+        {
+        "source_path_from_root": "/articles/machine-learning/how-to-use-batch-endpoint-sdk-v2.md",
+        "redirect_url": "/azure/machine-learning/batch-inference/how-to-use-batch-endpoint",
+        "redirect_document_id": false
+    },
+    {
+        "source_path_from_root": "/articles/machine-learning/how-to-use-batch-endpoints-studio.md",
+        "redirect_url": "/azure/machine-learning/batch-inference/how-to-use-batch-endpoint",
+        "redirect_document_id": false
+    },
+    {
+        "source_path_from_root": "/articles/machine-learning/how-to-troubleshoot-batch-endpoints.md",
+        "redirect_url": "/azure/machine-learning/batch-inference/how-to-troubleshoot-batch-endpoints",
+        "redirect_document_id": true
     }
 ]
-}
+}

articles/machine-learning/batch-inference/how-to-deploy-model-custom-output.md

@@ -32,8 +32,8 @@ In any of those cases, Batch Deployments allow you to take control of the output
 [!INCLUDE [basic cli prereqs](../../../includes/machine-learning-cli-prereqs.md)]
 
 * A model registered in the workspace. In this tutorial, we'll use an MLflow model. Particularly, we are using the *heart condition classifier* created in the tutorial [Using MLflow models in batch deployments](how-to-mlflow-batch.md).
-* You must have an endpoint already created. If you don't, follow the instructions at [Use batch endpoints for batch scoring](../how-to-use-batch-endpoint.md). This example assumes the endpoint is named `heart-classifier-batch`.
-* You must have a compute created where to deploy the deployment. If you don't, follow the instructions at [Create compute](../how-to-use-batch-endpoint.md#create-compute). This example assumes the name of the compute is `cpu-cluster`.
+* You must have an endpoint already created. If you don't, follow the instructions at [Use batch endpoints for batch scoring](how-to-use-batch-endpoint.md). This example assumes the endpoint is named `heart-classifier-batch`.
+* You must have a compute created where to deploy the deployment. If you don't, follow the instructions at [Create compute](how-to-use-batch-endpoint.md#create-compute). This example assumes the name of the compute is `cpu-cluster`.
 
 ## About this sample
 
@@ -145,7 +145,7 @@ Follow the next steps to create a deployment using the previous scoring script:
 2. MLflow models don't require you to indicate an environment or a scoring script when creating the deployments as it is created for you. However, in this case we are going to indicate a scoring script and environment since we want to customize how inference is executed.
 
    > [!NOTE]
-   > This example assumes you have an endpoint created with the name `heart-classifier-batch` and a compute cluster with name `cpu-cluster`. If you don't, please follow the steps in the doc [Use batch endpoints for batch scoring](../how-to-use-batch-endpoint.md).
+   > This example assumes you have an endpoint created with the name `heart-classifier-batch` and a compute cluster with name `cpu-cluster`. If you don't, please follow the steps in the doc [Use batch endpoints for batch scoring](how-to-use-batch-endpoint.md).
 
    # [Azure ML CLI](#tab/cli)
 

articles/machine-learning/batch-inference/how-to-image-processing-batch.md

@@ -1,5 +1,5 @@
 ---
-title: "Image processing tasks with batch deployments"
+title: "Image processing with batch deployments"
 titleSuffix: Azure Machine Learning
 description: Learn how to deploy a model in batch endpoints that process images
 services: machine-learning
@@ -13,7 +13,7 @@ ms.reviewer: larryfr
 ms.custom: devplatv2
 ---
 
-# Image processing tasks with batch deployments
+# Image processing with batch deployments
 
 [!INCLUDE [ml v2](../../../includes/machine-learning-dev-v2.md)]
 
@@ -23,8 +23,8 @@ Batch Endpoints can be used for processing tabular data, but also any other file
 
 [!INCLUDE [basic cli prereqs](../../../includes/machine-learning-cli-prereqs.md)]
 
-* You must have an endpoint already created. If you don't please follow the instructions at [Use batch endpoints for batch scoring](../how-to-use-batch-endpoint.md). This example assumes the endpoint is named `imagenet-classifier-batch`.
-* You must have a compute created where to deploy the deployment. If you don't please follow the instructions at [Create compute](../how-to-use-batch-endpoint.md#create-compute). This example assumes the name of the compute is `cpu-cluster`.
+* You must have an endpoint already created. If you don't please follow the instructions at [Use batch endpoints for batch scoring](how-to-use-batch-endpoint.md). This example assumes the endpoint is named `imagenet-classifier-batch`.
+* You must have a compute created where to deploy the deployment. If you don't please follow the instructions at [Create compute](how-to-use-batch-endpoint.md#create-compute). This example assumes the name of the compute is `cpu-cluster`.
 
 ## About the model used in the sample
 
@@ -169,7 +169,7 @@ One the scoring script is created, it's time to create a batch deployment for it
 1. Now, let create the deployment.
 
    > [!NOTE]
-   > This example assumes you have an endpoint created with the name `imagenet-classifier-batch` and a compute cluster with name `cpu-cluster`. If you don't, please follow the steps in the doc [Use batch endpoints for batch scoring](../how-to-use-batch-endpoint.md).
+   > This example assumes you have an endpoint created with the name `imagenet-classifier-batch` and a compute cluster with name `cpu-cluster`. If you don't, please follow the steps in the doc [Use batch endpoints for batch scoring](how-to-use-batch-endpoint.md).
 
    # [Azure ML CLI](#tab/cli)
 

articles/machine-learning/batch-inference/how-to-nlp-processing-batch.md

@@ -1,5 +1,5 @@
 ---
-title: "NLP tasks with batch deployments"
+title: "Text processing with batch deployments"
 titleSuffix: Azure Machine Learning
 description: Learn how to use batch deployments to process text and output results.
 services: machine-learning
@@ -13,7 +13,7 @@ ms.reviewer: larryfr
 ms.custom: devplatv2
 ---
 
-# NLP tasks with batch deployments
+# Text processing with batch deployments
 
 [!INCLUDE [cli v2](../../../includes/machine-learning-dev-v2.md)]
 
@@ -23,8 +23,8 @@ Batch Endpoints can be used for processing tabular data, but also any other file
 
 [!INCLUDE [basic cli prereqs](../../../includes/machine-learning-cli-prereqs.md)]
 
-* You must have an endpoint already created. If you don't please follow the instructions at [Use batch endpoints for batch scoring](../how-to-use-batch-endpoint.md). This example assumes the endpoint is named `text-summarization-batch`.
-* You must have a compute created where to deploy the deployment. If you don't please follow the instructions at [Create compute](../how-to-use-batch-endpoint.md#create-compute). This example assumes the name of the compute is `cpu-cluster`.
+* You must have an endpoint already created. If you don't please follow the instructions at [Use batch endpoints for batch scoring](how-to-use-batch-endpoint.md). This example assumes the endpoint is named `text-summarization-batch`.
+* You must have a compute created where to deploy the deployment. If you don't please follow the instructions at [Create compute](how-to-use-batch-endpoint.md#create-compute). This example assumes the name of the compute is `cpu-cluster`.
 
 ## About the model used in the sample
 
@@ -146,7 +146,7 @@ One the scoring script is created, it's time to create a batch deployment for it
 2. Now, let create the deployment.
 
    > [!NOTE]
-   > This example assumes you have an endpoint created with the name `text-summarization-batch` and a compute cluster with name `cpu-cluster`. If you don't, please follow the steps in the doc [Use batch endpoints for batch scoring](../how-to-use-batch-endpoint.md).
+   > This example assumes you have an endpoint created with the name `text-summarization-batch` and a compute cluster with name `cpu-cluster`. If you don't, please follow the steps in the doc [Use batch endpoints for batch scoring](how-to-use-batch-endpoint.md).
 
    # [Azure ML CLI](#tab/cli)
 

articles/machine-learning/batch-inference/how-to-secure-batch-endpoint.md

@@ -27,7 +27,7 @@ When deploying a machine learning model to a batch endpoint, you can secure thei
 All the batch endpoints created inside of secure workspace are deployed as private batch endpoints by default. Not further configuration is required.
 
 > [!IMPORTANT]
-> When working on a private link-enabled workspaces, batch endpoints can be created and managed using Azure Machine Learning studio. However, they can't be invoked from the UI in studio. Please use the Azure ML CLI v2 instead for job creation. For more details about how to use it see [Invoke the batch endpoint to start a batch scoring job](../how-to-use-batch-endpoint.md#invoke-the-batch-endpoint-to-start-a-batch-scoring-job).
+> When working on a private link-enabled workspaces, batch endpoints can be created and managed using Azure Machine Learning studio. However, they can't be invoked from the UI in studio. Please use the Azure ML CLI v2 instead for job creation. For more details about how to use it see [Invoke the batch endpoint to start a batch scoring job](how-to-use-batch-endpoint.md#invoke-the-batch-endpoint-to-start-a-batch-scoring-job).
 
 The following diagram shows how the networking looks like for batch endpoints when deployed in a private workspace:
 

articles/machine-learning/batch-inference/how-to-troubleshoot-batch-endpoints.md

@@ -0,0 +1,163 @@
+---
+title: "Troubleshooting batch endpoints"
+titleSuffix: Azure Machine Learning
+description: Learn how to troubleshoot and diagnostic errors with batch endpoints jobs
+services: machine-learning
+ms.service: machine-learning
+ms.subservice: core
+ms.topic: how-to
+author: santiagxf
+ms.author: fasantia
+ms.date: 10/10/2022
+ms.reviewer: larryfr
+ms.custom: devplatv2
+---
+
+# Troubleshooting batch endpoints
+
+[!INCLUDE [dev v2](../../../includes/machine-learning-dev-v2.md)]
+
+Learn how to troubleshoot and solve, or work around, common errors you may come across when using [batch endpoints](how-to-use-batch-endpoint.md) for batch scoring.
+
+## Understanding logs of a batch scoring job
+
+### Get logs
+
+After you invoke a batch endpoint using the Azure CLI or REST, the batch scoring job will run asynchronously. There are two options to get the logs for a batch scoring job.
+
+Option 1: Stream logs to local console
+
+You can run the following command to stream system-generated logs to your console. Only logs in the `azureml-logs` folder will be streamed.
+
+```azurecli
+az ml job stream -name <job_name>
+```
+
+Option 2: View logs in studio 
+
+To get the link to the run in studio, run: 
+
+```azurecli
+az ml job show --name <job_name> --query interaction_endpoints.Studio.endpoint -o tsv
+```
+
+1. Open the job in studio using the value returned by the above command. 
+1. Choose __batchscoring__
+1. Open the __Outputs + logs__ tab 
+1. Choose the log(s) you wish to review
+
+### Understand log structure
+
+There are two top-level log folders, `azureml-logs` and `logs`. 
+
+The file `~/azureml-logs/70_driver_log.txt` contains information from the controller that launches the scoring script.  
+
+Because of the distributed nature of batch scoring jobs, there are logs from several different sources. However, two combined files are created that provide high-level information: 
+
+- `~/logs/job_progress_overview.txt`: This file provides high-level information about the number of mini-batches (also known as tasks) created so far and the number of mini-batches processed so far. As the mini-batches end, the log records the results of the job. If the job failed, it will show the error message and where to start the troubleshooting.
+
+- `~/logs/sys/master_role.txt`: This file provides the principal node (also known as the orchestrator) view of the running job. This log provides information on task creation, progress monitoring, the job result.
+
+For a concise understanding of errors in your script there is:
+
+- `~/logs/user/error.txt`: This file will try to summarize the errors in your script.
+
+For more information on errors in your script, there is:
+
+- `~/logs/user/error/`: This file contains full stack traces of exceptions thrown while loading and running the entry script.
+
+When you need a full understanding of how each node executed the score script, look at the individual process logs for each node. The process logs can be found in the `sys/node` folder, grouped by worker nodes:
+
+- `~/logs/sys/node/<ip_address>/<process_name>.txt`: This file provides detailed info about each mini-batch as it's picked up or completed by a worker. For each mini-batch, this file includes:
+
+    - The IP address and the PID of the worker process. 
+    - The total number of items, the number of successfully processed items, and the number of failed items.
+    - The start time, duration, process time, and run method time.
+
+You can also view the results of periodic checks of the resource usage for each node. The log files and setup files are in this folder:
+
+- `~/logs/perf`: Set `--resource_monitor_interval` to change the checking interval in seconds. The default interval is `600`, which is approximately 10 minutes. To stop the monitoring, set the value to `0`. Each `<ip_address>` folder includes:
+
+    - `os/`: Information about all running processes in the node. One check runs an operating system command and saves the result to a file. On Linux, the command is `ps`.
+        - `%Y%m%d%H`: The sub folder name is the time to hour.
+            - `processes_%M`: The file ends with the minute of the checking time.
+    - `node_disk_usage.csv`: Detailed disk usage of the node.
+    - `node_resource_usage.csv`: Resource usage overview of the node.
+    - `processes_resource_usage.csv`: Resource usage overview of each process.
+
+### How to log in scoring script
+
+You can use Python logging in your scoring script. Logs are stored in `logs/user/stdout/<node_id>/processNNN.stdout.txt`. 
+
+```python
+import argparse
+import logging
+
+# Get logging_level
+arg_parser = argparse.ArgumentParser(description="Argument parser.")
+arg_parser.add_argument("--logging_level", type=str, help="logging level")
+args, unknown_args = arg_parser.parse_known_args()
+print(args.logging_level)
+
+# Initialize Python logger
+logger = logging.getLogger(__name__)
+logger.setLevel(args.logging_level.upper())
+logger.info("Info log statement")
+logger.debug("Debug log statement")
+```
+
+## Common issues
+
+The following section contains common problems and solutions you may see during batch endpoint development and consumption.
+
+### No module named 'azureml'
+
+__Reason__: Azure Machine Learning Batch Deployments require the package `azureml-core` to be installed.
+
+__Solution__: Add `azureml-core` to your conda dependencies file.
+
+### Output already exists
+
+__Reason__: Azure Machine Learning Batch Deployment can't overwrite the `predictions.csv` file generated by the output.
+
+__Solution__: If you are indicated an output location for the predictions, ensure the path leads to a non-existing file.
+
+### The run() function in the entry script had timeout for [number] times
+
+__Message logged__: `No progress update in [number] seconds. No progress update in this check. Wait [number] seconds since last update.`
+
+__Reason__: Batch Deployments can be configured with a `timeout` value that indicates the amount of time the deployment shall wait for a single batch to be processed. If the execution of the batch takes more than such value, the task is aborted. Tasks that are aborted can be retried up to a maximum of times that can also be configured. If the `timeout` occurs on each retry, then the deployment job fails. These properties can be configured for each deployment.
+
+__Solution__: Increase the `timemout` value of the deployment by updating the deployment. These properties are configured in the parameter `retry_settings`. By default, a `timeout=30` and `retries=3` is configured. When deciding the value of the `timeout`, take into consideration the number of files being processed on each batch and the size of each of those files. You can also decrease them to account for more mini-batches of smaller size and hence quicker to execute. 
+
+### Dataset initialization failed
+
+__Message logged__: Dataset initialization failed: UserErrorException: Message: Cannot mount Dataset(id='xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx', name='None', version=None). Source of the dataset is either not accessible or does not contain any data.
+
+__Reason__: The compute cluster where the deployment is running can't mount the storage where the data asset is located. The managed identity of the compute don't have permissions to perform the mount.
+
+__Solutions__: Ensure the identity associated with the compute cluster where your deployment is running has at least has at least [Storage Blob Data Reader](../../role-based-access-control/built-in-roles.md#storage-blob-data-reader) access to the storage account. Only storage account owners can [change your access level via the Azure portal](../../storage/blobs/assign-azure-role-data-access.md).
+
+### Data set node [code] references parameter dataset_param which doesn't have a specified value or a default value
+
+__Message logged__: Data set node [code] references parameter dataset_param which doesn't have a specified value or a default value.
+
+__Reason__: The input data asset provided to the batch endpoint isn't supported.
+
+__Solution__: Ensure you are providing a data input that is supported for batch endpoints.
+
+### User program failed with Exception: Run failed, please check logs for details
+
+__Message logged__: User program failed with Exception: Run failed, please check logs for details. You can check logs/readme.txt for the layout of logs.
+
+__Reason__: There was an error while running the `init()` or `run()` function of the scoring script.
+
+__Solution__: Go to __Outputs + Logs__ and open the file at `logs > user > error > 10.0.0.X > process000.txt`. You will see the error message generated by the `init()` or `run()` method.
+
+### There is no succeeded mini batch item returned from run()
+
+__Message logged__: There is no succeeded mini batch item returned from run(). Please check 'response: run()' in https://aka.ms/batch-inference-documentation.
+
+__Reason__: The batch endpoint failed to provide data in the expected format to the `run()` method. This may be due to corrupted files being read or incompatibility of the input data with the signature of the model (MLflow).
+
+__Solution__: To understand what may be happening, go to __Outputs + Logs__ and open the file at `logs > user > stdout > 10.0.0.X > process000.stdout.txt`. Look for error entries like `Error processing input file`. You should find there details about why the input file can't be correctly read.