Merge pull request #202803 from Blackmist/endpoint-logs

writing

Author: liljack17

articles/machine-learning/how-to-deploy-managed-online-endpoints.md

@@ -331,28 +331,9 @@ To view metrics and set alerts based on your SLA, complete the steps that are de
 
 ### (Optional) Integrate with Log Analytics
 
-The `get-logs` command provides only the last few hundred lines of logs from an automatically selected instance. However, Log Analytics provides a way to durably store and analyze logs. 
+The `get-logs` command provides only the last few hundred lines of logs from an automatically selected instance. However, Log Analytics provides a way to durably store and analyze logs. For more information on using logging, see [Monitor online endpoints](how-to-monitor-online-endpoints.md#logs)
 
-First, create a Log Analytics workspace by completing the steps in [Create a Log Analytics workspace in the Azure portal](../azure-monitor/logs/quick-create-workspace.md#create-a-workspace).
-
-Then, in the Azure portal:
-
-1. Go to the resource group.
-1. Select your endpoint.
-1. Select the **ARM resource page**.
-1. Select **Diagnostic settings**.
-1. Select **Add settings**.
-1. Select to enable sending console logs to the Log Analytics workspace.
-
-The logs might take up to an hour to connect. After an hour, send some scoring requests, and then check the logs by using the following steps:
-
-1. Open the Log Analytics workspace. 
-1. In the left menu, select **Logs**.
-1. Close the **Queries** dialog that automatically opens.
-1. Double-click **AmlOnlineEndpointConsoleLog**.
-1. Select **Run**.
-
-  [!INCLUDE [Email Notification Include](../../includes/machine-learning-email-notifications.md)]
+[!INCLUDE [Email Notification Include](../../includes/machine-learning-email-notifications.md)]
 
 ## Delete the endpoint and the deployment
 

articles/machine-learning/how-to-manage-quotas.md

@@ -119,7 +119,7 @@ Azure Machine Learning managed online endpoints have limits described in the fol
 
 <sup>3</sup> If you request a limit increase, be sure to calculate related limit increases you might need. For example, if you request a limit increase for requests per second, you might also want to compute the required connections and bandwidth limits and include these limit increases in the same request.
 
-To determine the current usage for an endpoint, [view the metrics](how-to-monitor-online-endpoints.md#view-metrics). 
+To determine the current usage for an endpoint, [view the metrics](how-to-monitor-online-endpoints.md#metrics). 
 
 To request an exception from the Azure Machine Learning product team, use the steps in the [Request quota increases](#request-quota-increases) section and provide the following information:
 

articles/machine-learning/how-to-monitor-online-endpoints.md

@@ -7,7 +7,7 @@ ms.service: machine-learning
 ms.author: larryfr
 author: blackmist
 ms.subservice: mlops
-ms.date: 06/01/2022
+ms.date: 06/27/2022
 ms.topic: conceptual
 ms.custom: how-to, devplatv2, event-tier1-build-2022
 ---
@@ -28,7 +28,7 @@ In this article you learn how to:
 - Deploy an Azure Machine Learning online endpoint.
 - You must have at least [Reader access](../role-based-access-control/role-assignments-portal.md) on the endpoint.
 
-## View metrics
+## Metrics
 
 Use the following steps to view metrics for an online endpoint or deployment:
 1. Go to the [Azure portal](https://portal.azure.com).
@@ -38,11 +38,11 @@ Use the following steps to view metrics for an online endpoint or deployment:
 
 1. In the left-hand column, select **Metrics**.
 
-## Available metrics
+### Available metrics
 
 Depending on the resource that you select, the metrics that you see will be different. Metrics are scoped differently for online endpoints and online deployments.
 
-### Metrics at endpoint scope
+#### Metrics at endpoint scope
 
 - Request Latency
 - Request Latency P50 (Request latency at the 50th percentile)
@@ -59,13 +59,13 @@ Split on the following dimensions:
 - Status Code
 - Status Code Class
 
-#### Bandwidth throttling
+**Bandwidth throttling**
 
 Bandwidth will be throttled if the limits are exceeded for _managed_ online endpoints (see managed online endpoints section in [Manage and increase quotas for resources with Azure Machine Learning](how-to-manage-quotas.md#azure-machine-learning-managed-online-endpoints)). To determine if requests are throttled:
 - Monitor the "Network bytes" metric
 - The response trailers will have the fields: `ms-azureml-bandwidth-request-delay-ms` and `ms-azureml-bandwidth-response-delay-ms`. The values of the fields are the delays, in milliseconds, of the bandwidth throttling.
 
-### Metrics at deployment scope
+#### Metrics at deployment scope
 
 - CPU Utilization Percentage
 - Deployment Capacity (the number of instances of the requested instance type)
@@ -78,11 +78,11 @@ Split on the following dimension:
 
 - InstanceId
 
-## Create a dashboard
+### Create a dashboard
 
 You can create custom dashboards to visualize data from multiple sources in the Azure portal, including the metrics for your online endpoint. For more information, see [Create custom KPI dashboards using Application Insights](../azure-monitor/app/tutorial-app-dashboards.md#add-custom-metric-chart).
 
-## Create an alert
+### Create an alert
 
 You can also create custom alerts to notify you of important status updates to your online endpoint:
 
@@ -97,6 +97,132 @@ You can also create custom alerts to notify you of important status updates to y
 1. Select **Add action groups** > **Create action groups** to specify what should happen when your alert is triggered.
 
 1. Choose **Create alert rule** to finish creating your alert.
+1. 
+## Logs
+
+There are three logs that can be enabled for online endpoints:
+
+* **AMLOnlineEndpointTrafficLog**: You could choose to enable traffic logs if you want to check the information of your request. Below are some cases: 
+
+    * If the response isn't 200, check the value of the column “ResponseCodeReason” to see what happened. Also check the reason in the "HTTPS status codes" section of the [Troubleshoot online endpoints](how-to-troubleshoot-online-endpoints.md#http-status-codes) article.
+
+    * You could check the response code and response reason of your model from the column “ModelStatusCode” and “ModelStatusReason”. 
+
+    * You want to check the duration of the request like total duration, the request/response duration, and the delay caused by the network throttling. You could check it from the logs to see the breakdown latency. 
+
+    * If you want to check how many requests or failed requests recently. You could also enable the logs. 
+
+* **AMLOnlineEndpointConsoleLog**: Contains logs that the containers output to the console. Below are some cases: 
+
+    * If the container fails to start, the console log may be useful for debugging. 
+
+    * Monitor container behavior and make sure that all requests are correctly handled. 
+
+    * Write request IDs in the console log. Joining the request ID, the AMLOnlineEndpointConsoleLog, and AMLOnlineEndpointTrafficLog in the Log Analytics workspace, you can trace a request from the network entry point of an online endpoint to the container.  
+
+    * You may also use this log for performance analysis in determining the time required by the model to process each request. 
+
+* **AMLOnlineEndpointEventLog**: Contains event information regarding the container’s life cycle. Currently, we provide information on the following types of events: 
+
+    | Name | Message |
+    | ----- | ----- | 
+    | BackOff | Back-off restarting failed container 
+    | Pulled | Container image "\<IMAGE\_NAME\>" already present on machine 
+    | Killing | Container inference-server failed liveness probe, will be restarted 
+    | Created | Created container image-fetcher 
+    | Created | Created container inference-server 
+    | Created | Created container model-mount 
+    | Unhealthy | Liveness probe failed: \<FAILURE\_CONTENT\> 
+    | Unhealthy | Readiness probe failed: \<FAILURE\_CONTENT\> 
+    | Started | Started container image-fetcher 
+    | Started | Started container inference-server 
+    | Started | Started container model-mount 
+    | Killing | Stopping container inference-server 
+    | Killing | Stopping container model-mount 
+
+### How to enable/disable logs
+
+> [!IMPORTANT]
+> Logging uses Azure Log Analytics. If you do not currently have a Log Analytics workspace, you can create one using the steps in [Create a Log Analytics workspace in the Azure portal](../azure-monitor/logs/quick-create-workspace.md#create-a-workspace).
+
+1. In the [Azure portal](https://portal.azure.com), go to the resource group that contains your endpoint and then select the endpoint.
+1. From the **Monitoring** section on the left of the page, select **Diagnostic settings** and then **Add settings**.
+1. Select the log categories to enable, select **Send to Log Analytics workspace**, and then select the Log Analytics workspace to use. Finally, enter a **Diagnostic setting name** and select **Save**.
+
+    :::image type="content" source="./media/how-to-monitor-online-endpoints/diagnostic-settings.png" alt-text="Screenshot of the diagnostic settings dialog.":::
+
+    > [!IMPORTANT]
+    > It may take up to an hour for the connection to the Log Analytics workspace to be enabled. Wait an hour before continuing with the next steps.
+    
+1. Submit scoring requests to the endpoint. This activity should create entries in the logs.
+1. From either the online endpoint properties or the Log Analytics workspace, select **Logs** from the left of the screen.
+1. Close the **Queries** dialog that automatically opens, and then double-click the **AmlOnlineEndpointConsoleLog**. If you don't see it, use the **Search** field.
+
+    :::image type="content" source="./media/how-to-monitor-online-endpoints/online-endpoints-log-queries.png" alt-text="Screenshot showing the log queries.":::
+
+1. Select **Run**.
+
+    :::image type="content" source="./media/how-to-monitor-online-endpoints/query-results.png" alt-text="Screenshots of the results after running a query.":::
+
+### Example queries
+
+You can find example queries on the __Queries__ tab while viewing logs. Search for __Online endpoint__ to find example queries.
+
+:::image type="content" source="./media/how-to-monitor-online-endpoints/example-queries.png" alt-text="Screenshot of the example queries.":::
+
+### Log column details 
+
+The following tables provide details on the data stored in each log:
+
+**AMLOnlineEndpointTrafficLog**
+
+| Field name | Description |
+| ---- | ---- |
+| Method | The requested method from client. 
+| Path | The requested path from client. 
+| SubscriptionId | The machine learning subscription ID of the online endpoint. 
+| WorkspaceId | The machine learning workspace ID of the online endpoint. 
+| EndpointName | The name of the online endpoint. 
+| DeploymentName | The name of the online deployment. 
+| Protocol | The protocol of the request. 
+| ResponseCode | The final response code returned to the client. 
+| ResponseCodeReason | The final response code reason returned to the client. 
+| ModelStatusCode | The response status code from model. 
+| ModelStatusReason | The response status reason from model. 
+| RequestPayloadSize | The total bytes received from the client. 
+| ResponsePayloadSize | The total bytes sent back to the client. 
+| UserAgent | The user-agent header of the request. 
+| XRequestId | The request ID generated by Azure Machine Learning for internal tracing. 
+| XMSClientRequestId | The tracking ID generated by the client. 
+| TotalDurationMs | Duration in milliseconds from the request start time to the last response byte sent back to the client. If the client disconnected, it measures from the start time to client disconnect time. 
+| RequestDurationMs | Duration in milliseconds from the request start time to the last byte of the request received from the client. 
+| ResponseDurationMs | Duration in milliseconds from the request start time to the first response byte read from the model. 
+| RequestThrottlingDelayMs | Delay in milliseconds in request data transfer due to network throttling. 
+| ResponseThrottlingDelayMs | Delay in milliseconds in response data transfer due to network throttling. 
+
+**AMLOnlineEndpointConsoleLog**
+
+| Field Name | Description |
+| ----- | ----- |
+| TimeGenerated | The timestamp (UTC) of when the log was generated. 
+| OperationName | The operation associated with log record. 
+| InstanceId | The ID of the instance that generated this log record. 
+| DeploymentName | The name of the deployment associated with the log record. 
+| ContainerName | The name of the container where the log was generated. 
+| Message | The content of the log. 
+
+**AMLOnlineEndpointEventLog**
+
+
+| Field Name | Description |
+| ----- | ----- |
+| TimeGenerated | The timestamp (UTC) of when the log was generated. 
+| OperationName | The operation associated with log record. 
+| InstanceId | The ID of the instance that generated this log record. 
+| DeploymentName | The name of the deployment associated with the log record. 
+| Name | The name of the event. 
+| Message | The content of the event. 
+
 
 
 ## Next steps

articles/machine-learning/how-to-safely-rollout-managed-endpoints.md

@@ -110,7 +110,7 @@ If you want to use a REST client to invoke the deployment directly without going
 Once you've tested your `green` deployment, you can copy (or 'mirror') a percentage of the live traffic to it. Mirroring traffic doesn't change results returned to clients. Requests still flow 100% to the blue deployment. The mirrored percentage of the traffic is copied and submitted to the `green` deployment so you can gather metrics and logging without impacting your clients. Mirroring is useful when you want to validate a new deployment without impacting clients. For example, to check if latency is within acceptable bounds and that there are no HTTP errors.
 
 > [!WARNING]
-> Mirroring traffic uses your [endpoint bandwidth quota](how-to-manage-quotas.md#azure-machine-learning-managed-online-endpoints) (default 5 MBPS). Your endpoint bandwidth will be throttled if you exceed the allocated quota. For information on monitoring bandwidth throttling, see [Monitor managed online endpoints](how-to-monitor-online-endpoints.md#bandwidth-throttling).
+> Mirroring traffic uses your [endpoint bandwidth quota](how-to-manage-quotas.md#azure-machine-learning-managed-online-endpoints) (default 5 MBPS). Your endpoint bandwidth will be throttled if you exceed the allocated quota. For information on monitoring bandwidth throttling, see [Monitor managed online endpoints](how-to-monitor-online-endpoints.md#metrics-at-endpoint-scope).
 
 The following command mirrors 10% of the traffic to the `green` deployment: