Skip to content

Commit 7f82b33

Browse files
authored
Merge pull request #109393 from mamccrea/patch-43
Stream Analytics: Update stream-analytics-troubleshoot-query.md
2 parents f28b2d6 + a3bb724 commit 7f82b33

File tree

1 file changed

+15
-6
lines changed

1 file changed

+15
-6
lines changed

articles/stream-analytics/stream-analytics-troubleshoot-query.md

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -6,20 +6,25 @@ ms.author: sidram
66
ms.reviewer: mamccrea
77
ms.service: stream-analytics
88
ms.topic: conceptual
9-
ms.date: 12/07/2018
9+
ms.date: 03/31/2020
1010
ms.custom: seodec18
1111
---
1212

1313
# Troubleshoot Azure Stream Analytics queries
1414

1515
This article describes common issues with developing Stream Analytics queries and how to troubleshoot them.
1616

17+
This article describes common issues with developing Azure Stream Analytics queries, how to troubleshoot query issues, and how to correct the issues. Many troubleshooting steps require diagnostic logs to be enabled for your Stream Analytics job. If you do not have diagnostic logs enabled, see [Troubleshoot Azure Stream Analytics by using diagnostics logs](stream-analytics-job-diagnostic-logs.md).
18+
1719
## Query is not producing expected output
20+
1821
1. Examine errors by testing locally:
22+
1923
- On Azure portal, on the **Query** tab, select **Test**. Use the downloaded sample data to [test the query](stream-analytics-test-query.md). Examine any errors and attempt to correct them.
2024
- You can also [test your query locally](stream-analytics-live-data-local-testing.md) using Azure Stream Analytics tools for Visual Studio or [Visual Studio Code](visual-studio-code-local-run-live-input.md).
2125

22-
2. [Debug queries step by step locally using job diagram](debug-locally-using-job-diagram.md) in Azure Stream Analytics tools for Visual Studio. The job diagram is to show how data flows from input sources (Event Hub, IoT Hub, etc.) through multiple query steps and finally output to sinks. Each query step is mapped to a temporary result set defined in the script using WITH statement. You can view the data as well as metrics in each query step in each intermediate result set to find the source of the issue.
26+
2. [Debug queries step by step locally using job diagram](debug-locally-using-job-diagram.md) in Azure Stream Analytics tools for Visual Studio. The job diagram shows how data flows from input sources (Event Hub, IoT Hub, etc.) through multiple query steps and finally to output sinks. Each query step is mapped to a temporary result set defined in the script using the WITH statement. You can view the data, as well as metrics, in each intermediate result set to find the source of the issue.
27+
2328
![Job diagram preview result](./media/debug-locally-using-job-diagram/preview-result.png)
2429

2530
3. If you use [**Timestamp By**](https://docs.microsoft.com/stream-analytics-query/timestamp-by-azure-stream-analytics), verify that the events have timestamps greater than the [job start time](stream-analytics-out-of-order-and-late-events.md).
@@ -28,20 +33,24 @@ This article describes common issues with developing Stream Analytics queries an
2833
- A [**WHERE**](https://docs.microsoft.com/stream-analytics-query/where-azure-stream-analytics) clause in the query filtered out all events, preventing any output from being generated.
2934
- A [**CAST**](https://docs.microsoft.com/stream-analytics-query/cast-azure-stream-analytics) function fails, causing the job to fail. To avoid type cast failures, use [**TRY_CAST**](https://docs.microsoft.com/stream-analytics-query/try-cast-azure-stream-analytics) instead.
3035
- When you use window functions, wait for the entire window duration to see an output from the query.
31-
- The timestamp for events precedes the job start time and, therefore, events are being dropped.
36+
- The timestamp for events precedes the job start time and events are dropped.
37+
- [**JOIN**](https://docs.microsoft.com/stream-analytics-query/join-azure-stream-analytics) conditions don't match. If there are no matches, there will be zero output.
3238

33-
5. Ensure event ordering policies are configured as expected. Go to the **Settings** and select [**Event Ordering**](stream-analytics-out-of-order-and-late-events.md). The policy is *not* applied when you use the **Test** button to test the query. This result is one difference between testing in-browser versus running the job in production.
39+
5. Ensure event ordering policies are configured as expected. Go to **Settings** and select [**Event Ordering**](stream-analytics-out-of-order-and-late-events.md). The policy is *not* applied when you use the **Test** button to test the query. This result is one difference between testing in-browser versus running the job in production.
3440

3541
6. Debug by using audit and diagnostic logs:
3642
- Use [Audit Logs](../azure-resource-manager/resource-group-audit.md), and filter to identify and debug errors.
3743
- Use [job diagnostic logs](stream-analytics-job-diagnostic-logs.md) to identify and debug errors.
3844

39-
## Job is consuming too many Streaming Units
45+
## Resource utilization is high
46+
4047
Ensure you take advantage of parallelization in Azure Stream Analytics. You can learn to [scale with query parallelization](stream-analytics-parallelization.md) of Stream Analytics jobs by configuring input partitions and tuning the analytics query definition.
4148

4249
## Debug queries progressively
4350

44-
In real-time data processing, knowing what the data looks like in the middle of the query can be helpful. Because inputs or steps of an Azure Stream Analytics job can be read multiple times, you can write extra SELECT INTO statements. Doing so outputs intermediate data into storage and lets you inspect the correctness of the data, just as *watch variables* do when you debug a program.
51+
In real-time data processing, knowing what the data looks like in the middle of the query can be helpful. You can see this using the job diagram in Visual Studio. If you don't have Visual Studio, you can take additional steps to output intermediate data.
52+
53+
Because inputs or steps of an Azure Stream Analytics job can be read multiple times, you can write extra SELECT INTO statements. Doing so outputs intermediate data into storage and lets you inspect the correctness of the data, just as *watch variables* do when you debug a program.
4554

4655
The following example query in an Azure Stream Analytics job has one stream input, two reference data inputs, and an output to Azure Table Storage. The query joins data from the event hub and two reference blobs to get the name and category information:
4756

0 commit comments

Comments
 (0)