|
| 1 | +--- |
| 2 | +title: Diagnostic logging |
| 3 | +titleSuffix: Azure Cognitive Services |
| 4 | +description: This guide provides step-by-step instructions to enable diagnostic logging for an Azure Cognitive Service. These logs provide rich, frequent data about the operation of a resource that are used for issue identification and debugging. |
| 5 | +services: cognitive-services |
| 6 | +author: erhopf |
| 7 | +manager: cgronlun |
| 8 | +ms.service: cognitive-services |
| 9 | +ms.topic: article |
| 10 | +ms.date: 06/14/2019 |
| 11 | +ms.author: erhopf |
| 12 | +--- |
| 13 | + |
| 14 | +# Enable diagnostic logging for Azure Cognitive Services |
| 15 | + |
| 16 | +This guide provides step-by-step instructions to enable diagnostic logging for an Azure Cognitive Service. These logs provide rich, frequent data about the operation of a resource that are used for issue identification and debugging. Before you continue, you must have an Azure account with a subscription to at least one Cognitive Service, such as [Bing Web Search](https://docs.microsoft.com/azure/cognitive-services/bing-web-search/overview), [Speech Services](https://docs.microsoft.com/azure/cognitive-services/speech-service/overview), or [LUIS](https://docs.microsoft.com/azure/cognitive-services/luis/what-is-luis). |
| 17 | + |
| 18 | +## Prerequisites |
| 19 | + |
| 20 | +To enable diagnostic logging, you'll need somewhere to store your log data. This tutorial uses Azure Storage and Log Analytics. |
| 21 | + |
| 22 | +* [Azure storage](https://docs.microsoft.com/azure/monitoring-and-diagnostics/monitoring-archive-diagnostic-logs) - Retains diagnostic logs for policy audit, static analysis, or backup. The storage account does not have to be in the same subscription as the resource emitting logs as long as the user who configures the setting has appropriate RBAC access to both subscriptions. |
| 23 | +* [Log Analytics](https://docs.microsoft.com/azure/monitoring-and-diagnostics/monitor-stream-diagnostic-logs-log-analytics) - A flexible log search and analytics tool that allows for analysis of raw logs generated by an Azure resource. |
| 24 | + |
| 25 | +> [!NOTE] |
| 26 | +> Additional configuration options are available. To learn more, see [Collect and consume log data from your Azure resources](https://docs.microsoft.com/azure/monitoring-and-diagnostics/monitoring-overview-of-diagnostic-logs). |
| 27 | +
|
| 28 | +## Enable diagnostic log collection |
| 29 | + |
| 30 | +Let's start by enabling diagnostic logging using the Azure portal. |
| 31 | + |
| 32 | +> [!NOTE] |
| 33 | +> To enable this feature using PowerShell or the Azure CLI, use the instructions provided in [Collect and consume log data from your Azure resources](https://docs.microsoft.com/azure/monitoring-and-diagnostics/monitoring-overview-of-diagnostic-logs#how-to-enable-collection-of-diagnostic-logs). |
| 34 | +
|
| 35 | +1. Navigate to the Azure portal. Then locate and select a Cognitive Services resource. For example, your subscription to Bing Web Search. |
| 36 | +2. Next, from the left-hand navigation menu, locate **Monitoring** and select **Diagnostic settings**. This screen contains all previously created diagnostic settings for this resource. |
| 37 | +3. If there is a previously created resource that you'd like to use, you can select it now. Otherwise, select **+ Add diagnostic setting**. |
| 38 | +4. Enter a name for the setting. Then select **Archive to a storage account** and **Send to log Analytics**. |
| 39 | +5. When prompted to configure, select the storage account and OMS workspace that you'd like to use to store you diagnostic logs. **Note**: If you don't have a storage account or OMS workspace, follow the prompts to create one. |
| 40 | +6. Select **Audit**, **RequestResponse**, and **AllMetrics**. Then set the retention period for your diagnostic log data. If a retention policy is set to zero, events for that log category are stored indefinitely. |
| 41 | +7. Click **Save**. |
| 42 | + |
| 43 | +It can take up to two hours before logging data is available to query and analyze. So don't worry if you don't see anything right away. |
| 44 | + |
| 45 | +## View and export diagnostic data from Azure Storage |
| 46 | + |
| 47 | +Azure Storage is a robust object storage solution that is optimized for storing large amounts of unstructured data. In this section, you'll learn to query your storage account for total transactions over a 30-day timeframe and export the data to excel. |
| 48 | + |
| 49 | +1. From the Azure portal, locate the Azure Storage resource that you created in the last section. |
| 50 | +2. From the left-hand navigation menu, locate **Monitoring** and select **Metrics**. |
| 51 | +3. Use the available drop-downs to configure your query. For this example, let's set the time range to **Last 30 days** and the metric to **Transaction**. |
| 52 | +4. When the query is complete, you'll see a visualization of transaction over the last 30 days. To export this data, use the **Export to Excel** button located at the top of the page. |
| 53 | + |
| 54 | +Learn more about what you can do with diagnostic data in [Azure Storage](https://docs.microsoft.com/azure/storage/blobs/storage-blobs-introduction). |
| 55 | + |
| 56 | +## View logs in Log Analytics |
| 57 | + |
| 58 | +Follow these instructions to explore log analytics data for your resource. |
| 59 | + |
| 60 | +1. From the Azure portal, locate and select **Log Analytics** from the left-hand navigation menu. |
| 61 | +2. Locate and select the resource you created when enabling diagnostics. |
| 62 | +3. Under **General**, locate and select **Logs**. From this page, you can run queries against your logs. |
| 63 | + |
| 64 | +### Sample queries |
| 65 | + |
| 66 | +Here are a few basic Kusto queries you can use to explore your log data. |
| 67 | + |
| 68 | +Run this query for all diagnostic logs from Azure Cognitive Services for a specified time period: |
| 69 | + |
| 70 | +```kusto |
| 71 | +AzureDiagnostics |
| 72 | +| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES" |
| 73 | +``` |
| 74 | + |
| 75 | +Run this query to see the 10 most recent logs: |
| 76 | + |
| 77 | +```kusto |
| 78 | +AzureDiagnostics |
| 79 | +| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES" |
| 80 | +| take 10 |
| 81 | +``` |
| 82 | + |
| 83 | +Run this query to group operations by **Resource**: |
| 84 | + |
| 85 | +```kusto |
| 86 | +AzureDiagnostics |
| 87 | +| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES" | |
| 88 | +summarize count() by Resource |
| 89 | +``` |
| 90 | +Run this query to find the average time it takes to perform an operation: |
| 91 | + |
| 92 | +```kusto |
| 93 | +AzureDiagnostics |
| 94 | +| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES" |
| 95 | +| summarize avg(DurationMs) |
| 96 | +by OperationName |
| 97 | +``` |
| 98 | + |
| 99 | +Run this query to view the volume of operations over time split by OperationName with counts binned for every 10s. |
| 100 | + |
| 101 | +```kusto |
| 102 | +AzureDiagnostics |
| 103 | +| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES" |
| 104 | +| summarize count() |
| 105 | +by bin(TimeGenerated, 10s), OperationName |
| 106 | +| render areachart kind=unstacked |
| 107 | +``` |
| 108 | + |
| 109 | +## Next steps |
| 110 | + |
| 111 | +* To understand how to enable logging, and also the metrics and log categories that are supported by the various Azure services, read both the [Overview of metrics](https://docs.microsoft.com/azure/monitoring-and-diagnostics/monitoring-overview-metrics) in Microsoft Azure and [Overview of Azure Diagnostic Logs](https://docs.microsoft.com/azure/azure-monitor/platform/diagnostic-logs-overview) articles. |
| 112 | +* Read these articles to learn about event hubs: |
| 113 | + * [What is Azure Event Hubs?](https://docs.microsoft.com/azure/event-hubs/event-hubs-what-is-event-hubs) |
| 114 | + * [Get started with Event Hubs](https://docs.microsoft.com/azure/event-hubs/event-hubs-csharp-ephcs-getstarted) |
| 115 | +* Read [Download metrics and diagnostic logs from Azure Storage](https://docs.microsoft.com/azure/storage/blobs/storage-quickstart-blobs-dotnet#download-blobs). |
| 116 | +* Read [Understand log searches in Azure Monitor logs](https://docs.microsoft.com/azure/log-analytics/log-analytics-log-search-new). |
0 commit comments