|
| 1 | +--- |
| 2 | +title: Query data in Azure Monitor using Azure Data Explorer (preview) |
| 3 | +description: Use Azure Data Explorer to perform cross product queries between Azure Data Explorer, Log Analytics workspaces and classic Application Insights applications in Azure Monitor. |
| 4 | +author: orens |
| 5 | +ms.author: bwren |
| 6 | +ms.reviewer: bwren |
| 7 | +ms.subservice: logs |
| 8 | +ms.topic: conceptual |
| 9 | +ms.date: 10/13/2020 |
| 10 | + |
| 11 | +--- |
| 12 | + |
| 13 | +# Query data in Azure Monitor using Azure Data Explorer (preview) |
| 14 | +The Azure Data Explorer proxy cluster enables you to perform cross product queries between Azure Data Explorer, Log Analytics workspaces and classic Application Insights applications in Azure Monitor. You can map Log Analytics workspaces in Azure Monitor or classic Application Insights apps as proxy clusters. You can then query the proxy cluster using Azure Data Explorer tools and refer to it in a cross cluster query. The article shows how to connect to a proxy cluster, add a proxy cluster to Azure Data Explorer Web UI, and run queries against your Log Analytics workspaces or classic Application Insights apps from Azure Data Explorer. |
| 15 | + |
| 16 | +The following diagram shows the Azure Data Explorer proxy flow: |
| 17 | + |
| 18 | +:::image type="content" source="media/azure-data-explorer-monitor-proxy/azure-data-explorer-monitor-proxy-flow.png" alt-text="Azure data explorer proxy flow."::: |
| 19 | + |
| 20 | + |
| 21 | +> [!NOTE] |
| 22 | +> The Azure Data Explorer proxy is in public preview. [Connect to the proxy](#connect-to-the-proxy) to enable the proxy feature for your clusters. |
| 23 | +
|
| 24 | +## Connect to the proxy |
| 25 | +To connect your Log Analytics workspace or classic Application Insights app, open the[Azure Data Explorer Web UI](https://dataexplorer.azure.com/clusters). Verify your Azure Data Explorer native cluster (such as *help* cluster) appears on the left menu before you connect to your Log Analytics or Application Insights cluster. |
| 26 | + |
| 27 | +:::image type="content" source="media/azure-data-explorer-monitor-proxy/azure-data-explorer-web-ui-help-cluster.png" alt-text="Azure Data Explorer native cluster."::: |
| 28 | + |
| 29 | +Click **Add Cluster** and then add the URL of the Log Analytics or Application Insights cluster in one of the following formats. |
| 30 | + |
| 31 | +* For Log Analytics: `https://ade.loganalytics.io/subscriptions/<subscription-id>/resourcegroups/<resource-group-name>/providers/microsoft.operationalinsights/workspaces/<workspace-name>` |
| 32 | +* For Application Insights: `https://ade.applicationinsights.io/subscriptions/<subscription-id>/resourcegroups/<resource-group-name>/providers/microsoft.insights/components/<ai-app-name>` |
| 33 | + |
| 34 | +Click **Add** to make the connection. |
| 35 | + |
| 36 | +:::image type="content" source="media/azure-data-explorer-monitor-proxy/azure-monitor-proxy-add-cluster.png" alt-text="Add cluster."::: |
| 37 | + |
| 38 | +> [!NOTE] |
| 39 | +> If you add a connection to more than one proxy cluster, give each a different name. Otherwise they'll all have the same name in the left pane. |
| 40 | +
|
| 41 | +After the connection is established, your Log Analytics or Application Insights cluster will appear in the left pane with your native Azure Data Explorer cluster. |
| 42 | + |
| 43 | +:::image type="content" source="media/azure-data-explorer-monitor-proxy/azure-monitor-azure-data-explorer-clusters.png" alt-text="Log Analytics and Azure Data Explorer clusters."::: |
| 44 | + |
| 45 | +> [!NOTE] |
| 46 | +> The number of Azure Monitor workspaces that can be mapped is limited to 100. |
| 47 | +
|
| 48 | +## Create queries using Azure Monitor data |
| 49 | + |
| 50 | +You can run the queries using client tools that support Kusto queries, such as: Kusto Explorer, Azure Data Explorer Web UI, Jupyter Kqlmagic, Flow, PowerQuery, PowerShell, Jarvis, Lens, and REST API. |
| 51 | + |
| 52 | +> [!NOTE] |
| 53 | +> The Azure Data Explorer proxy feature is used for data retrieval only. For more information, see [Function supportability](#function-supportability). |
| 54 | +
|
| 55 | +> [!TIP] |
| 56 | +> * Database name should have the same name as the resource specified in the proxy cluster. Names are case sensitive. |
| 57 | +> * In cross cluster queries, make sure that the naming of Application Insights apps and Log Analytics workspaces is correct. |
| 58 | +> * If names contain special characters, they're replaced by URL encoding in the proxy cluster name. |
| 59 | +> * If names include characters that don't meet [KQL identifier name rules](https://docs.microsoft.com/azure/data-explorer/kusto/query/schema-entities/entity-names), they are replaced by the dash **-** character. |
| 60 | +
|
| 61 | +### Direct query from your Log Analytics or Application Insights proxy cluster |
| 62 | + |
| 63 | +Run queries on your Log Analytics or Application Insights cluster. Verify that your cluster is selected in the left pane. |
| 64 | + |
| 65 | +```kusto |
| 66 | +Perf | take 10 // Demonstrate query through the proxy on the Log Analaytics workspace |
| 67 | +``` |
| 68 | + |
| 69 | +:::image type="content" source="media/azure-data-explorer-monitor-proxy/azure-monitor-proxy-query-la.png" alt-text="Query Log Analytics workspace."::: |
| 70 | + |
| 71 | +### Cross query of your Log Analytics or Application Insights proxy cluster and the Azure Data Explorer native cluster |
| 72 | + |
| 73 | +When you run cross cluster queries from the proxy, verify your Azure Data Explorer native cluster is selected in the left pane. The following examples demonstrate combining Azure Data Explorer cluster tables using the [union](/azure/data-explorer/kusto/query/unionoperator) operator with a Log Analytics workspace. |
| 74 | + |
| 75 | +```kusto |
| 76 | +union StormEvents, cluster('https://ade.loganalytics.io/subscriptions/<subscription-id>/resourcegroups/<resource-group-name>/providers/microsoft.operationalinsights/workspaces/<workspace-name>').database('<workspace-name>').Perf |
| 77 | +| take 10 |
| 78 | +``` |
| 79 | + |
| 80 | +```kusto |
| 81 | +let CL1 = 'https://ade.loganalytics.io/subscriptions/<subscription-id>/resourcegroups/<resource-group-name>/providers/microsoft.operationalinsights/workspaces/<workspace-name>'; |
| 82 | +union <Azure Data Explorer table>, cluster(CL1).database(<workspace-name>).<table name> |
| 83 | +``` |
| 84 | +Using the [`join` operator](/azure/data-explorer/kusto/query/joinoperator?pivots=azuredataexplorer), instead of union, may require a [hint](/azure/data-explorer/kusto/query/joinoperator?pivots=azuredataexplorer#join-hints) to run it on an Azure Data Explorer native cluster (and not on the proxy). |
| 85 | + |
| 86 | +### Join data from an Azure Data Explorer cluster in one tenant with an Azure Monitor resource in another |
| 87 | + |
| 88 | +Cross-tenant queries aren't supported by Azure Data Explorer proxy. You are signed in to a single tenant for running the query spanning both resources. |
| 89 | + |
| 90 | +If the Azure Data Explorer resource is in Tenant 'A' and Log Analytics workspace is in Tenant 'B' use one of the following two methods: |
| 91 | + |
| 92 | +- Azure Data Explorer allows you to add roles for principals in different tenants. Add your user ID in Tenant 'B' as an authorized user on the Azure Data Explorer cluster. Validate the *['TrustedExternalTenant'](https://docs.microsoft.com/powershell/module/az.kusto/update-azkustocluster)* property on the Azure Data Explorer cluster contains Tenant 'B'. Run the cross-query fully in Tenant 'B'. |
| 93 | + |
| 94 | +- Use [Lighthouse](/azure/lighthouse/) to project the Azure Monitor resource into Tenant 'A'. |
| 95 | + |
| 96 | +### Connect to Azure Data Explorer clusters from different tenants |
| 97 | + |
| 98 | +Kusto Explorer automatically signs you into the tenant to which the user account originally belongs. To access resources in other tenants with the same user account, the `tenantId` has to be explicitly specified in the connection string: |
| 99 | +`Data Source=https://ade.applicationinsights.io/subscriptions/SubscriptionId/resourcegroups/ResourceGroupName;Initial Catalog=NetDefaultDB;AAD Federated Security=True;Authority ID=`**TenantId** |
| 100 | + |
| 101 | +## Function supportability |
| 102 | + |
| 103 | +The Azure Data Explorer proxy cluster supports functions for both Log Analytics and Application Insights. This capability enables cross-cluster queries to reference an Azure Monitor tabular function directly. |
| 104 | + |
| 105 | +The following commands are supported by the proxy: |
| 106 | + |
| 107 | +* `.show functions` |
| 108 | +* `.show function {FunctionName}` |
| 109 | +* `.show database {DatabaseName} schema as json` |
| 110 | + |
| 111 | +The following image depicts an example of querying a tabular function from the Azure Data Explorer Web UI. To use the function, run the name in the Query window. |
| 112 | + |
| 113 | +:::image type="content" source="media/azure-data-explorer-monitor-proxy/azure-monitor-proxy-function-query.png" alt-text="Query a tabular function from Azure Data Explorer Web UI."::: |
| 114 | + |
| 115 | +> [!NOTE] |
| 116 | +> Azure Monitor only supports tabular functions, which don't support parameters. |
| 117 | +
|
| 118 | +## Additional syntax examples |
| 119 | + |
| 120 | +The following syntax options are available when calling the Log Analytics or Application Insights clusters: |
| 121 | + |
| 122 | +|Syntax Description |Application Insights |Log Analytics | |
| 123 | +|----------------|---------|---------| |
| 124 | +| Database within a cluster that contains only the defined resource in this subscription (**recommended for cross cluster queries**) | cluster(`https://ade.applicationinsights.io/subscriptions/<subscription-id>/resourcegroups/<resource-group-name>/providers/microsoft.insights/components/<ai-app-name>').database('<ai-app-name>`) | cluster(`https://ade.loganalytics.io/subscriptions/<subscription-id>/resourcegroups/<resource-group-name>/providers/microsoft.operationalinsights/workspaces/<workspace-name>').database('<workspace-name>`) | |
| 125 | +| Cluster that contains all apps/workspaces in this subscription | cluster(`https://ade.applicationinsights.io/subscriptions/<subscription-id>`) | cluster(`https://ade.loganalytics.io/subscriptions/<subscription-id>`) | |
| 126 | +|Cluster that contains all apps/workspaces in the subscription and are members of this resource group | cluster(`https://ade.applicationinsights.io/subscriptions/<subscription-id>/resourcegroups/<resource-group-name>`) | cluster(`https://ade.loganalytics.io/subscriptions/<subscription-id>/resourcegroups/<resource-group-name>`) | |
| 127 | +|Cluster that contains only the defined resource in this subscription | cluster(`https://ade.applicationinsights.io/subscriptions/<subscription-id>/resourcegroups/<resource-group-name>/providers/microsoft.insights/components/<ai-app-name>`) | cluster(`https://ade.loganalytics.io/subscriptions/<subscription-id>/resourcegroups/<resource-group-name>/providers/microsoft.operationalinsights/workspaces/<workspace-name>`) | |
| 128 | + |
| 129 | +## Next steps |
| 130 | + |
| 131 | +- Read more about the [data structure of Log Analytics workspaces and Application Insights](data-platform-logs.md). |
| 132 | +- Learn to [write queries in Azure Data Explorer](https://docs.microsoft.com/azure/data-explorer/write-queries). |
0 commit comments