Merge pull request #217427 from KevinConanMSFT/patch-1

Update sql-data-warehouse-manage-monitor.md

Author: v-regandowner

articles/synapse-analytics/sql-data-warehouse/sql-data-warehouse-manage-monitor.md

@@ -1,20 +1,19 @@
 ---
-title: Monitor your dedicated SQL pool workload using DMVs 
+title: Monitor your dedicated SQL pool workload using DMVs
 description: Learn how to monitor your Azure Synapse Analytics dedicated SQL pool workload and query execution using DMVs.
 author: WilliamDAssafMSFT
-manager: craigg
+ms.author: wiassaf
+ms.reviewer: kecona
+ms.date: 11/09/2022
 ms.service: synapse-analytics
+ms.subservice: sql-dw
 ms.topic: conceptual
-ms.subservice: sql-dw 
-ms.date: 11/15/2021
-ms.author: wiassaf
-ms.reviewer: wiassaf
 ms.custom: synapse-analytics
 ---
 
 # Monitor your Azure Synapse Analytics dedicated SQL pool workload using DMVs
 
-This article describes how to use Dynamic Management Views (DMVs) to monitor your workload including investigating query execution in SQL pool.
+This article describes how to use Dynamic Management Views (DMVs) to monitor your workload including investigating query execution in a dedicated SQL pool.
 
 ## Permissions
 
@@ -26,7 +25,7 @@ GRANT VIEW DATABASE STATE TO myuser;
 
 ## Monitor connections
 
-All logins to your data warehouse are logged to [sys.dm_pdw_exec_sessions](/sql/relational-databases/system-dynamic-management-views/sys-dm-pdw-exec-sessions-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true).  This DMV contains the last 10,000 logins.  The session_id is the primary key and is assigned sequentially for each new logon.
+All logins to your data warehouse are logged to [sys.dm_pdw_exec_sessions](/sql/relational-databases/system-dynamic-management-views/sys-dm-pdw-exec-sessions-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true).  This DMV contains the last 10,000 logins.  The `session_id` is the primary key and is assigned sequentially for each new login.
 
 ```sql
 -- Other Active Connections
@@ -35,9 +34,9 @@ SELECT * FROM sys.dm_pdw_exec_sessions where status <> 'Closed' and session_id <
 
 ## Monitor query execution
 
-All queries executed on SQL pool are logged to [sys.dm_pdw_exec_requests](/sql/relational-databases/system-dynamic-management-views/sys-dm-pdw-exec-requests-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true).  This DMV contains the last 10,000 queries executed.  The request_id uniquely identifies each query and is the primary key for this DMV.  The request_id is assigned sequentially for each new query and is prefixed with QID, which stands for query ID.  Querying this DMV for a given session_id shows all queries for a given logon.
+All queries executed on SQL pool are logged to [sys.dm_pdw_exec_requests](/sql/relational-databases/system-dynamic-management-views/sys-dm-pdw-exec-requests-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true).  This DMV contains the last 10,000 queries executed.  The `request_id` uniquely identifies each query and is the primary key for this DMV.  The `request_id` is assigned sequentially for each new query and is prefixed with QID, which stands for query ID.  Querying this DMV for a given `session_id` shows all queries for a given login.
 
-> [!NOTE]
+> [!NOTE]  
 > Stored procedures use multiple Request IDs.  Request IDs are assigned in sequential order.
 
 Here are steps to follow to investigate query execution plans and times for a particular query.
@@ -56,14 +55,13 @@ ORDER BY submit_time DESC;
 SELECT TOP 10 *
 FROM sys.dm_pdw_exec_requests
 ORDER BY total_elapsed_time DESC;
-
 ```
 
 From the preceding query results, **note the Request ID** of the query that you would like to investigate.
 
 Queries in the **Suspended** state can be queued due to a large number of active running queries. These queries also appear in the [sys.dm_pdw_waits](/sql/relational-databases/system-dynamic-management-views/sys-dm-pdw-waits-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true). In that case, look for waits such as UserConcurrencyResourceType. For information on concurrency limits, see [Memory and concurrency limits](memory-concurrency-limits.md) or [Resource classes for workload management](resource-classes-for-workload-management.md). Queries can also wait for other reasons such as for object locks.  If your query is waiting for a resource, see [Investigating queries waiting for resources](#monitor-waiting-queries) further down in this article.
 
-To simplify the lookup of a query in the [sys.dm_pdw_exec_requests](/sql/relational-databases/system-dynamic-management-views/sys-dm-pdw-exec-requests-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true) table, use [LABEL](/sql/t-sql/queries/option-clause-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true) to assign a comment to your query, which can be looked up in the sys.dm_pdw_exec_requests view.
+To simplify the lookup of a query in the [sys.dm_pdw_exec_requests](/sql/relational-databases/system-dynamic-management-views/sys-dm-pdw-exec-requests-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true) table, use [LABEL](/sql/t-sql/queries/option-clause-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true) to assign a comment to your query, which can be looked up in the `sys.dm_pdw_exec_requests` view.
 
 ```sql
 -- Query with Label
@@ -94,7 +92,7 @@ ORDER BY step_index;
 
 When a DSQL plan is taking longer than expected, the cause can be a complex plan with many DSQL steps or just one step taking a long time.  If the plan is many steps with several move operations, consider optimizing your table distributions to reduce data movement. The [Table distribution](sql-data-warehouse-tables-distribute.md) article explains why data must be moved to solve a query. The article also explains some distribution strategies to minimize data movement.
 
-To investigate further details about a single step, the *operation_type* column of the long-running query step and note the **Step Index**:
+To investigate further details about a single step, the `operation_type` column of the long-running query step and note the **Step Index**:
 
 * Proceed with Step 3 for **SQL operations**: OnOperation, RemoteOperation, ReturnOperation.
 * Proceed with Step 4 for **Data Movement operations**: ShuffleMoveOperation, BroadcastMoveOperation, TrimMoveOperation, PartitionMoveOperation, MoveOperation, CopyOperation.
@@ -132,8 +130,8 @@ SELECT * FROM sys.dm_pdw_dms_workers
 WHERE request_id = 'QID####' AND step_index = 2;
 ```
 
-* Check the *total_elapsed_time* column to see if a particular distribution is taking significantly longer than others for data movement.
-* For the long-running distribution, check the *rows_processed* column to see if the number of rows being moved from that distribution is significantly larger than others. If so, this finding might indicate skew of your underlying data. One cause for data skew is distributing on a column with many NULL values (whose rows will all land in the same distribution). Prevent slow queries by avoiding distribution on these types of columns or filtering your query to eliminate NULLs when possible. 
+* Check the `total_elapsed_time` column to see if a particular distribution is taking significantly longer than others for data movement.
+* For the long-running distribution, check the `rows_processed` column to see if the number of rows being moved from that distribution is significantly larger than others. If so, this finding might indicate skew of your underlying data. One cause for data skew is distributing on a column with many NULL values (whose rows will all land in the same distribution). Prevent slow queries by avoiding distribution on these types of columns or filtering your query to eliminate NULLs when possible.
 
 If the query is running, you can use [DBCC PDW_SHOWEXECUTIONPLAN](/sql/t-sql/database-console-commands/dbcc-pdw-showexecutionplan-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true) to retrieve the SQL Server estimated plan from the SQL Server plan cache for the currently running SQL Step within a particular distribution.
 
@@ -155,10 +153,10 @@ If you discover that your query is not making progress because it is waiting for
 -- Replace request_id with value from Step 1.
 
 SELECT waits.session_id,
-      waits.request_id,  
+      waits.request_id,
       requests.command,
       requests.status,
-      requests.start_time,  
+      requests.start_time,
       waits.type,
       waits.state,
       waits.object_type,
@@ -174,11 +172,11 @@ If the query is actively waiting on resources from another query, then the state
 
 ## Monitor tempdb
 
-Tempdb is used to hold intermediate results during query execution. High utilization of the tempdb database can lead to slow query performance. For every DW100c configured, 399 GB of tempdb space is allocated (DW1000c would have 3.99 TB of total tempdb space).  Below are tips for monitoring tempdb usage and for decreasing tempdb usage in your queries.
+The `tempdb` database is used to hold intermediate results during query execution. High utilization of the `tempdb` database can lead to slow query performance. For every DW100c configured, 399 GB of `tempdb` space is allocated (DW1000c would have 3.99 TB of total `tempdb` space).  Below are tips for monitoring `tempdb` usage and for decreasing `tempdb` usage in your queries.
 
-### Monitoring tempdb with views
+### Monitor tempdb with views
 
-To monitor tempdb usage, first install the [microsoft.vw_sql_requests](https://github.com/Microsoft/sql-data-warehouse-samples/blob/master/solutions/monitoring/scripts/views/microsoft.vw_sql_requests.sql) view from the [Microsoft Toolkit for SQL pool](https://github.com/Microsoft/sql-data-warehouse-samples/tree/master/solutions/monitoring). You can then execute the following query to see the tempdb usage per node for all executed queries:
+To monitor `tempdb` usage, first install the [microsoft.vw_sql_requests](https://github.com/Microsoft/sql-data-warehouse-samples/blob/master/solutions/monitoring/scripts/views/microsoft.vw_sql_requests.sql) view from the [Microsoft Toolkit for SQL pool](https://github.com/Microsoft/sql-data-warehouse-samples/tree/master/solutions/monitoring). You can then execute the following query to see the `tempdb` usage per node for all executed queries:
 
 ```sql
 -- Monitor tempdb
@@ -212,11 +210,14 @@ WHERE DB_NAME(ssu.database_id) = 'tempdb'
 ORDER BY sr.request_id;
 ```
 
-If you have a query that is consuming a large amount of memory or have received an error message related to the allocation of tempdb, it could be due to a very large [CREATE TABLE AS SELECT (CTAS)](/sql/t-sql/statements/create-table-as-select-azure-sql-data-warehouse) or [INSERT SELECT](/sql/t-sql/statements/insert-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true) statement running that is failing in the final data movement operation. This can usually be identified as a ShuffleMove operation in the distributed query plan right before the final INSERT SELECT.  Use [sys.dm_pdw_request_steps](/sql/relational-databases/system-dynamic-management-views/sys-dm-pdw-request-steps-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true) to monitor ShuffleMove operations.
+> [!NOTE]  
+> Data Movement uses a hidden database called `QTABLE`.  When that database is filled, the query will also return an error message about `tempdb` being out of space.  Details about `QTABLE` are not returned in the above query.
 
-The most common mitigation is to break your CTAS or INSERT SELECT statement into multiple load statements so the data volume will not exceed the 2TB per node tempdb limit (when at or above DW500c). You can also scale your cluster to a larger size which will spread the tempdb size across more nodes reducing the tempdb on each individual node.
+If you have a query that is consuming a large amount of memory or have received an error message related to the allocation of `tempdb`, it could be due to a very large [CREATE TABLE AS SELECT (CTAS)](/sql/t-sql/statements/create-table-as-select-azure-sql-data-warehouse) or [INSERT SELECT](/sql/t-sql/statements/insert-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true) statement running that is failing in the final data movement operation. This can usually be identified as a ShuffleMove operation in the distributed query plan right before the final INSERT SELECT.  Use [sys.dm_pdw_request_steps](/sql/relational-databases/system-dynamic-management-views/sys-dm-pdw-request-steps-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true) to monitor ShuffleMove operations.
 
-In addition to CTAS and INSERT SELECT statements, large, complex queries running with insufficient memory can spill into tempdb causing queries to fail.  Consider running with a larger [resource class](resource-classes-for-workload-management.md) to avoid spilling into tempdb.
+The most common mitigation is to break your CTAS or INSERT SELECT statement into multiple load statements so that the data volume will not exceed the 399 GB per 100DWUc `tempdb` limit. You can also scale your cluster to a larger size to increase how much `tempdb` space you have.
+
+In addition to CTAS and INSERT SELECT statements, large, complex queries running with insufficient memory can spill into `tempdb` causing queries to fail.  Consider running with a larger [resource class](resource-classes-for-workload-management.md) to avoid spilling into `tempdb`.
 
 ## Monitor memory
 
@@ -282,7 +283,6 @@ GROUP BY t.pdw_node_id, nod.[type]
 The following query provides an approximate estimate of the progress of your load. The query only shows files currently being processed.
 
 ```sql
-
 -- To track bytes and files
 SELECT
     r.command,
@@ -305,57 +305,54 @@ ORDER BY
 
 ## Monitor query blockings
 
-The following query provides the top 500 blocked queries in the environment. 
+The following query provides the top 500 blocked queries in the environment.
 
 ```sql
-
 --Collect the top blocking
-SELECT 
+SELECT
     TOP 500 waiting.request_id AS WaitingRequestId,
     waiting.object_type AS LockRequestType,
     waiting.object_name AS ObjectLockRequestName,
     waiting.request_time AS ObjectLockRequestTime,
     blocking.session_id AS BlockingSessionId,
     blocking.request_id AS BlockingRequestId
-FROM 
+FROM
     sys.dm_pdw_waits waiting
     INNER JOIN sys.dm_pdw_waits blocking
-    ON waiting.object_type = blocking.object_type 
-    AND waiting.object_name = blocking.object_name 
-WHERE 
-    waiting.state = 'Queued' 
+    ON waiting.object_type = blocking.object_type
+    AND waiting.object_name = blocking.object_name
+WHERE
+    waiting.state = 'Queued'
     AND blocking.state = 'Granted'
-ORDER BY 
+ORDER BY
     ObjectLockRequestTime ASC;
-    
-``` 
+```
 
 ## Retrieve query text from waiting and blocking queries
 
 The following query provides the query text and identifier for the waiting and blocking queries to easily troubleshoot.
 
 ```sql
-
 -- To retrieve query text from waiting and blocking queries
 
 SELECT waiting.session_id AS WaitingSessionId,
        waiting.request_id AS WaitingRequestId,
-       COALESCE(waiting_exec_request.command,waiting_exec_request.command2) AS WaitingExecRequestText, 
+       COALESCE(waiting_exec_request.command,waiting_exec_request.command2) AS WaitingExecRequestText,
        blocking.session_id AS BlockingSessionId,
        blocking.request_id AS BlockingRequestId,
        COALESCE(blocking_exec_request.command,blocking_exec_request.command2) AS BlockingExecRequestText,
        waiting.object_name AS Blocking_Object_Name,
        waiting.object_type AS Blocking_Object_Type,
        waiting.type AS Lock_Type,
        waiting.request_time AS Lock_Request_Time,
-       datediff(ms, waiting.request_time, getdate())/1000.0 AS Blocking_Time_sec 
+       datediff(ms, waiting.request_time, getdate())/1000.0 AS Blocking_Time_sec
 FROM sys.dm_pdw_waits waiting
        INNER JOIN sys.dm_pdw_waits blocking
        ON waiting.object_type = blocking.object_type
-       AND waiting.object_name = blocking.object_name 
+       AND waiting.object_name = blocking.object_name
        INNER JOIN sys.dm_pdw_exec_requests blocking_exec_request
        ON blocking.request_id = blocking_exec_request.request_id
-       INNER JOIN sys.dm_pdw_exec_requests waiting_exec_request 
+       INNER JOIN sys.dm_pdw_exec_requests waiting_exec_request
        ON waiting.request_id = waiting_exec_request.request_id
 WHERE waiting.state = 'Queued'
        AND blocking.state = 'Granted'
@@ -364,4 +361,4 @@ ORDER BY Lock_Request_Time DESC;
 
 ## Next steps
 
-For more information about DMVs, see [System views](../sql/reference-tsql-system-views.md).
+- For more information about DMVs, see [System views](../sql/reference-tsql-system-views.md).