Merge pull request #113446 from bwren/am-diagnostic-metric

Azure Monitor diagnostic extension configure in portal

Author: v-albemi

articles/azure-monitor/faq.md

@@ -93,6 +93,11 @@ Many resource providers are automatically registered, but you may need to manual
 ### Why am I am getting no access error message when opening Log Analytics from a VM? 
 To view VM Logs, you need to be granted with read permission to the workspaces that stores the VM logs. In these cases, your administrator must grant you with to permissions in Azure.
 
+## Metrics
+
+### Why are metrics from the guest OS of my Azure virtual machine not showing up in Metrics explorer?
+[Platform metrics](insights/monitor-azure-resource.md#monitoring-data) are collected automatically for Azure resources. You must perform some configuration though to collect metrics from the guest OS of a virtual machine. For a Windows VM, install the diagnostic extension and configure the Azure Monitor sink as described in [Install and configure Windows Azure diagnostics extension (WAD)](platform/diagnostics-extension-windows-install.md). For Linux, install the Telegraf agent as described in [Collect custom metrics for a Linux VM with the InfluxData Telegraf agent](platform/collect-custom-metrics-linux-telegraf.md).
+
 ## Alerts
 
 ### What is an alert in Azure Monitor?

articles/azure-monitor/insights/media/monitor-vm-azure/metrics.png

@@ -6,7 +6,7 @@ ms.subservice: logs
 ms.topic: conceptual
 author: bwren
 ms.author: bwren
-ms.date: 03/17/2020
+ms.date: 05/05/2020
 
 ---
 
@@ -20,7 +20,7 @@ This article describes how to use Azure Monitor to collect and analyze monitorin
 ## Differences from other Azure resources
 [Monitoring Azure resources with Azure Monitor](monitor-azure-resource.md) describes the monitoring data generated by Azure resources and how you can use the features of Azure Monitor to analyze and alert on this data. You can collect and act on the same monitoring data from Azure virtual machines with the following differences:
 
-- [Platform metrics](../platform/data-platform-metrics.md) are collected automatically for virtual machines but only for the [virtual machine host](#monitoring-data). You need an agent to collect performance data from the guest operating system. 
+-  [Platform metrics](../platform/data-platform-metrics.md) are collected automatically for virtual machines but only for the [virtual machine host](#monitoring-data). You need an agent to collect performance data from the guest operating system. 
 - Virtual machines don't generate [resource logs](../platform/platform-logs-overview.md) which provide insight into operations that were performed within an Azure resource. You use an agent to collect log data from the guest operating system.
 - You can create [diagnostic settings](../platform/diagnostic-settings.md) for a virtual machine to send platform metrics to other destinations such as storage and event hubs, but you can't configure these diagnostic settings in the Azure portal. 
 
@@ -117,7 +117,6 @@ az monitor diagnostic-settings create \
 --resource /subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/my-resource-group/providers/Microsoft.Compute/virtualMachines/my-vm \
 --metrics '[{"category": "AllMetrics","enabled": true}]' \
 --workspace /subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourcegroups/my-resource-group/providers/microsoft.operationalinsights/workspaces/my-workspace
-
 ```
 
 ## Monitoring in the Azure portal 
@@ -145,12 +144,13 @@ Once you configure collection of monitoring data for a virtual machine, you have
 ## Analyzing metric data
 You can analyze metrics for virtual machines using metrics explorer by opening **Metrics** from the virtual machine's menu. See [Getting started with Azure Metrics Explorer](../platform/metrics-getting-started.md) for details on using this tool. 
 
-There are two namespaces used by virtual machines for metrics:
+There are three namespaces used by virtual machines for metrics:
 
-| Namespace | Description |
-|:---|:---|
-| Virtual Machine Host | Host metrics automatically collected for all Azure virtual machines. Detailed list of metrics at [Microsoft.Compute/virtualMachines](../platform/metrics-supported.md#microsoftcomputevirtualmachines). |
-| Virtual Machine Guest | Guest operating system metrics collected from virtual machines with diagnostics extension installed and configured to send to Azure Monitor sink. |
+| Namespace | Description | Requirement |
+|:---|:---|:---|
+| Virtual Machine Host | Host metrics automatically collected for all Azure virtual machines. Detailed list of metrics at [Microsoft.Compute/virtualMachines](../platform/metrics-supported.md#microsoftcomputevirtualmachines). | Collected automatically with no configuration required. |
+| Guest (classic) | Limited set of guest operating system and application performance data. Available in metrics explorer but not other Azure Monitor features such as metric alerts.  | [Diagnostic extension](../platform/diagnostics-extension-overview.md) installed. Data is read from Azure storage.  |
+| Virtual Machine Guest | Guest operating system and application performance data available to all Azure Monitor features using metrics. | For Windows, [diagnostic extension installed](../platform/diagnostics-extension-overview.md) installed with Azure Monitor sink enabled. For Linux, [Telegraf agent installed](../platform/collect-custom-metrics-linux-telegraf.md). |
 
 ![Metrics](media/monitor-vm-azure/metrics.png)
 

articles/azure-monitor/insights/monitor-vm-azure.md

@@ -1,23 +1,20 @@
 ---
-title: Collect Windows VM metrics in Azure Monitor with template
-description: Send guest OS metrics to the Azure Monitor metric store using a Resource Manager template for a Windows virtual machine
+title: Collect Windows VM metrics in Azure Monitor  with template
+description: Send guest OS metrics to the Azure Monitor metric database store by using a Resource Manager template for a Windows virtual machine 
 author: anirudhcavale
 services: azure-monitor
 
 ms.topic: conceptual
-ms.date: 09/24/2018
-ms.author: ancav
+ms.date: 05/04/2020
+ms.author: bwren
 ms.subservice: metrics
 ---
-# Send Guest OS metrics to the Azure Monitor metric store using a Resource Manager template for a Windows virtual machine
+# Send guest OS metrics to the Azure Monitor metric store by using an Azure Resource Manager template for a Windows virtual machine
+Performance data from the guest OS of Azure virtual machines is not collected automatically like other [platform metrics](../insights/monitor-azure-resource.md#monitoring-data). Install the Azure Monitor [diagnostics extension](diagnostics-extension-overview.md) to collect guest OS metrics into the metrics database so it can be used with all features of Azure Monitor Metrics, including near-real time alerting, charting, routing, and access from a REST API. This article describes the process for sending Guest OS performance metrics for a Windows virtual machine to the metrics database using a Resource Manager template. 
 
-[!INCLUDE [updated-for-az](../../../includes/updated-for-az.md)]
-
-By using the Azure Monitor [Diagnostics extension](diagnostics-extension-overview.md), you can collect metrics and logs from the guest operating system (Guest OS) that's running as part of a virtual machine, cloud service, or Service Fabric cluster. The extension can send telemetry to [many different locations.](https://docs.microsoft.com/azure/monitoring/monitoring-data-collection?toc=/azure/azure-monitor/toc.json)
-
-This article describes the process for sending Guest OS performance metrics for a Windows virtual machine to the Azure Monitor data store. Starting with Diagnostics version 1.11, you can write metrics directly to the Azure Monitor metrics store, where standard platform metrics are already collected.
+> [!NOTE]
+> For details on configuring the diagnostics extension to collect guest OS metrics using the Azure portal, see [Install and configure Windows Azure diagnostics extension (WAD)](diagnostics-extension-windows-install.md).
 
-Storing them in this location allows you to access the same actions for platform metrics. Actions include near-real time alerting, charting, routing, and access from a REST API and more. In the past, the Diagnostics extension wrote to Azure Storage, but not to the Azure Monitor data store.
 
 If you're new to Resource Manager templates, learn about [template deployments](../../azure-resource-manager/management/overview.md) and their structure and syntax.
 

articles/azure-monitor/platform/collect-custom-metrics-guestos-resource-manager-vm.md

@@ -15,6 +15,14 @@ Azure Diagnostics extension is an [agent in Azure Monitor](agents-overview.md) t
 > [!NOTE]
 > Azure Diagnostics extension is one of the agents available to collect monitoring data from the guest operating system of compute resources. See [Overview of the Azure Monitor agents ](agents-overview.md) for a description of the different agents and guidance on selecting the appropriate agents for your requirements.
 
+## Primary scenarios
+The primary scenarios addressed by the diagnostics extension are:
+
+- Collect guest metrics into Azure Monitor Metrics.
+- Send guest logs and metrics to Azure storage for archiving.
+- Send guest logs and metrics to Azure event hubs to send outside of Azure.
+
+
 ## Comparison to Log Analytics agent
 The Log Analytics agent in Azure Monitor can also be used to collect monitoring data from the guest operating system of virtual machines. You may choose to use either or both depending on your requirements. See [Overview of the Azure Monitor agents](agents-overview.md) for a detailed comparison of the Azure Monitor agents. 
 

articles/azure-monitor/platform/diagnostics-extension-overview.md

@@ -10,35 +10,60 @@ ms.date: 02/17/2020
 ms.author: bwren
 ---
 # Install and configure Windows Azure diagnostics extension (WAD)
-Azure diagnostics extension is an agent in Azure Monitor that collects monitoring data from the guest operating system and workloads of Azure virtual machines and other compute resources. This article provides details on installing and configuring the Windows diagnostics extension and a description of how the data is stored in and Azure Storage account.
+[Azure diagnostics extension](diagnostics-extension-overview.md) is an agent in Azure Monitor that collects monitoring data from the guest operating system and workloads of Azure virtual machines and other compute resources. This article provides details on installing and configuring the Windows diagnostics extension and a description of how the data is stored in and Azure Storage account.
 
 The diagnostics extension is implemented as a [virtual machine extension](../../virtual-machines/extensions/overview.md) in Azure, so it supports the same installation options using Resource Manager templates, PowerShell, and CLI. See [Virtual machine extensions and features for Windows](../../virtual-machines/extensions/features-windows.md) for details on installing and maintaining virtual machine extensions.
 
+## Overview
+When you configure Windows Azure the diagnostics extension, you must specify a storage account where all specified data will be sent. You can optionally add one for more *data sinks* to send the data to different locations.
+
+- Azure Monitor sink - Send guest performance data to Azure Monitor Metrics.
+- Event hub sink - Send guest performance and log data to Azure event hubs to forward outside of Azure. This sink cannot be configured in the Azure portal.
+
+
 ## Install with Azure portal
 You can install and configure the diagnostics extension on an individual virtual machine in the Azure portal which provides you an interface as opposed to working directly with the configuration. When you enable the diagnostics extension, it will automatically use a default configuration with the most common performance counters and events. You can modify this default configuration according to your specific requirements.
 
 > [!NOTE]
-> There are diagnostics extension settings you cannot configure using the Azure portal including sending data to Azure Event Hubs. You must use one of the other configuration methods for these settings.
+> The following describe the most common settings for the diagnostics extension. For details on all of the configuration options, see [Windows diagnostics extension schema](diagnostics-extension-schema-windows.md).
 
 1. Open the menu for a virtual machine in the Azure portal.
+
 2. Click on **Diagnostic settings** in the **Monitoring** section of the VM menu.
+
 3. Click **Enable guest-level monitoring** if the diagnostics extension hasn't already been enabled.
-4. A new Azure Storage account will be created for the VM with the name will be based on the name of the resource group for the VM. You can attach the VM to another storage account by selecting the **Agent** tab.
 
-![Diagnostic settings](media/diagnostics-extension-windows-install/diagnostic-settings.png)
+   ![Enable monitoring](media/diagnostics-extension-windows-install/enable-monitoring.png)
+
+4. A new Azure Storage account will be created for the VM with the name will be based on the name of the resource group for the VM, and a default set of guest performance counters and logs will be selected.
+
+   ![Diagnostic settings](media/diagnostics-extension-windows-install/diagnostic-settings.png)
+
+5. In the **Performance counters** tab, select the guest metrics you would like to collect from this virtual machine. Use the **Custom** setting for more advanced selection.
 
+   ![Performance counters](media/diagnostics-extension-windows-install/performance-counters.png)
 
-You can modify the default configuration once the diagnostics extension has been enabled. The following table describes the options you can modify in the different tabs. Some options have a **Custom** command which allows you to specify more detailed configuration; see [Windows diagnostics extension schema](diagnostics-extension-schema-windows.md) for details on different settings.
+6. In the **Logs** tab, select the logs to collect from the virtual machine. Logs can be sent to storage or event hubs, but not to Azure Monitor. Use the [Log Analytics agent](log-analytics-agent.md) to collect guest logs to Azure Monitor.
 
-| Tab | Description |
-|:---|:---|
-| Overview | Displays the current configuration with links to the other tabs. |
-| Performance counters | Select the performance counters to collect and the sample rate for each.  |
-| Logs | Select the log data to collect. This includes Windows Event logs, IIS logs, .NET application logs and ETW events.  |
-| Crash dumps | Enable crash dump for different processes. |
-| Sinks | Enable data sinks to send data to destinations in addition to Azure Storage.<br>Azure Monitor - Sends performance data to Azure Monitor Metrics.<br>Application Insights - Send data to an Application Insights application. |
-| Agent | Modify the following configuration for the agent:<br>- Change the storage account.<br>- Specify the maximum local disk used for the agent.<br>- Configure logs for the health of the agent itself.|
+   ![Logs](media/diagnostics-extension-windows-install/logs.png)
 
+7. In the **Crash dumps** tab, specify any processes to collect memory dumps after a crash. The data will be written to the storage account for the diagnostic setting, and you can optionally specify a blob container.
+
+   ![Crash dumps](media/diagnostics-extension-windows-install/crash-dumps.png)
+
+8. In the **Sinks** tab, specify whether to send the data to locations other than Azure storage. If you select **Azure Monitor**, guest performance data will be sent to Azure Monitor Metrics. You cannot configure the event hubs sink using the Azure portal.
+
+   ![Sinks](media/diagnostics-extension-windows-install/sinks.png)
+   
+   If you have not enabled a System Assigned Identity configured for your virtual machine, you may see the below warning when you save a configuration with the Azure Monitor sink. Click on the banner to enable the system assigned identity.
+   
+   ![Managed entity](media/diagnostics-extension-windows-install/managed-entity.png)
+
+9. In the **Agent**, you can change the storage account, set the disk quota, and specify whether to collect diagnostic infrastructure logs.  
+
+   ![Agent](media/diagnostics-extension-windows-install/agent.png)
+
+10. Click **Save** to save the configuration. 
 
 > [!NOTE]
 > While the configuration for diagnostics extension can be formatted in either JSON or XML, any configuration done in the Azure portal will always be stored as JSON. If you use XML with another configuration method and then change your configuration with the Azure portal, the settings will be changed to JSON.
@@ -68,6 +93,7 @@ The protected settings are defined in the [PrivateConfig element](diagnostics-ex
     "storageAccountEndPoint": "https://mystorageaccount.blob.core.windows.net"
 }
 ```
+
 The public settings are defined in the [Public element](diagnostics-extension-schema-windows.md#publicconfig-element) of the configuration schema. Following is a minimal example of a public settings file that enables collection of diagnostic infrastructure logs,  a single performance counter, and a single event log. See [Example configuration](diagnostics-extension-schema-windows.md#publicconfig-element) for complete details of the public settings.
 
 ```JSON
@@ -178,5 +204,5 @@ Several tools are available to view the data after it is transferred to storage.
 * [Microsoft Azure Storage Explorer](../../vs-azure-tools-storage-manage-with-storage-explorer.md) is a standalone app that enables you to easily work with Azure Storage data on Windows, OSX, and Linux.
 * [Azure Management Studio](https://www.cerebrata.com/products/azure-management-studio/introduction) includes Azure Diagnostics Manager which allows you to view, download and manage the diagnostics data collected by the applications running on Azure.
 
-## Next Steps
+## Next steps
 - See [Send data from Windows Azure diagnostics extension to Event Hubs](diagnostics-extension-stream-event-hubs.md) for details on forwarding monitoring data to Azure Event Hubs.