Skip to content

Commit 6e9553f

Browse files
authored
Merge pull request #100965 from ronortloff/master
WLM Monitoring adds and updates
2 parents 14182eb + eb8dd67 commit 6e9553f

File tree

6 files changed

+88
-3
lines changed

6 files changed

+88
-3
lines changed

articles/sql-data-warehouse/TOC.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -225,6 +225,8 @@
225225
href: sql-data-warehouse-workload-importance.md
226226
- name: Workload isolation
227227
href: sql-data-warehouse-workload-isolation.md
228+
- name: Workload management monitoring
229+
href: sql-data-warehouse-workload-management-portal-monitor.md
228230
- name: Memory & concurrency limits
229231
href: memory-concurrency-limits.md
230232
- name: Resource classes
25.5 KB
Loading
20.3 KB
Loading

articles/sql-data-warehouse/sql-data-warehouse-workload-isolation.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ manager: craigg
77
ms.service: sql-data-warehouse
88
ms.topic: conceptual
99
ms.subservice: workload-management
10-
ms.date: 11/27/2019
10+
ms.date: 01/13/2020
1111
ms.author: rortloff
1212
ms.reviewer: jrasnick
1313
ms.custom: seo-lt-2019
@@ -85,3 +85,4 @@ Access to resources in the shared pool is allocated on an [importance](sql-data-
8585
- [Quickstart: configure workload isolation](quickstart-configure-workload-isolation-tsql.md)
8686
- [CREATE WORKLOAD GROUP](/sql/t-sql/statements/create-workload-group-transact-sql?view=azure-sqldw-latest)
8787
- [Convert resource classes to workload groups](sql-data-warehouse-how-to-convert-resource-classes-workload-groups.md).
88+
- [Workload Management Portal Monitoring](sql-data-warehouse-workload-management-portal-monitor.md).
Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
---
2+
title: Workload management portal monitoring
3+
description: Guidance for workload management portal monitoring in Azure Synapse Analytics.
4+
services: sql-data-warehouse
5+
author: ronortloff
6+
manager: craigg
7+
ms.service: sql-data-warehouse
8+
ms.topic: conceptual
9+
ms.subservice: workload-management
10+
ms.date: 01/13/2020
11+
ms.author: rortloff
12+
ms.reviewer: jrasnick
13+
ms.custom: seo-lt-2019
14+
---
15+
16+
# Azure Synapse Analytics – Workload Management Portal Monitoring (Preview)
17+
This article explains how to monitor [workload group](sql-data-warehouse-workload-isolation.md#workload-groups) resource utilization and query activity.
18+
For details on how to configure the Azure Metrics Explorer see the [Getting started with Azure Metrics Explorer](../azure-monitor/platform/metrics-getting-started.md) article. See the [Resource utilization](sql-data-warehouse-concept-resource-utilization-query-activity.md#resource-utilization) section in Azure SQL Data Warehouse Monitoring documentation for details on how to monitor system resource consumption.
19+
There are two different categories of workload group metrics provided for monitoring workload management: resource allocation and query activity. These metrics can be split and filtered by workload group. The metrics can be split and filtered based on if they are system defined (resource class workload groups) or user-defined (created by user with [CREATE WORKLOAD GROUP](https://docs.microsoft.com/sql/t-sql/statements/create-workload-group-transact-sql?view=azure-sqldw-latest) syntax).
20+
21+
## Workload management metric definitions
22+
23+
|Metric Name |Description |Aggregation Type |
24+
|-------------------------------|-------------|-----------------|
25+
|Effective cap resource percent | *Effective cap resource percent* is a hard limit on the percentage of resources accessible by the workload group, taking into account *Effective min resource percentage* allocated for other workload groups. The *Effective cap resource percent* metric is configured using the `CAP_PERCENTAGE_RESOURCE` parameter in the [CREATE WORKLOAD GROUP](https://docs.microsoft.com/sql/t-sql/statements/create-workload-group-transact-sql?view=azure-sqldw-latest) syntax. The effective value is described here.<br><br>For example if a workload group `DataLoads` is created with `CAP_PERCENTAGE_RESOURCE` = 100 and another workload group is created with an Effective min resource percentage of 25%, the *Effective cap resource percent* for the `DataLoads` workload group is 75%.<br><br>The *Effective cap resource percent* determines the upper bound of concurrency (and thus potential throughput) a workload group can achieve. If additional throughput is needed beyond what is currently reported by the *Effective cap resource percent* metric, either increase the `CAP_PERCENTAGE_RESOURCE`, decrease the `MIN_PERCENTAGE_RESOURCE` of other workload groups or scale up the instance to add more resources. Decreasing the `REQUEST_MIN_RESOURCE_GRANT_PERCENT` can increase concurrency, but may not increase overall throughput.| Min, Avg, Max |
26+
|Effective min resource percent |*Effective min resource percent* is the minimum percentage of resources reserved and isolated for the workload group taking into account the service level minimum. The Effective min resource percent metric is configured using the `MIN_PERCENTAGE_RESOURCE` parameter in the [CREATE WORKLOAD GROUP](https://docs.microsoft.com/sql/t-sql/statements/create-workload-group-transact-sql?view=azure-sqldw-latest) syntax. The effective value is described [here](https://docs.microsoft.com/sql/t-sql/statements/create-workload-group-transact-sql?view=azure-sqldw-latest#effective-values).<br><br>Use the Sum aggregation type when this metric is unfiltered and unsplit to monitor the total workload isolation configured on the system.<br><br>The *Effective min resource percent* determines the lower bound of guaranteed concurrency (and thus guaranteed throughput) a workload group can achieve. If additional guaranteed resources are needed beyond what is currently reported by the *Effective min resource percent* metric, increase the `MIN_PERCENTAGE_RESOURCE` parameter configured for the workload group. Decreasing the `REQUEST_MIN_RESOURCE_GRANT_PERCENT` can increase concurrency, but may not increase overall throughput. |Min, Avg, Max|
27+
|Workload group active queries |This metric reports the active queries within the workload group. Using this metric unfiltered and unsplit displays all active queries running on the system.|Sum |
28+
|Workload group allocation by max resource percent |This metric displays the percentage allocation of resources relative to the *Effective cap resource percent* per workload group. This metric provides the effective utilization of the workload group.<br><br>Consider a workload group `DataLoads` with an *Effective cap resource percent* of 75% and a `REQUEST_MIN_RESOURCE_GRANT_PERCENT` configured at 25%. The *Workload group allocation by max resource percent* value filtered to `DataLoads` would be 33% (25% / 75%) if a single query were running in this workload group.<br><br>Use this metric to identify a workload group’s utilization. A value close to 100% indicates all resources available to the workload group are being used. Additionally, the *Workload group queued queries metric* for the same workload group showing a value greater than zero would indicate the workload group would utilize additional resources if allocated. Conversely, if this metric is consistently low and the *Workload group active queries* is low the workload group is not being utilized. This situation is especially problematic if *Effective cap resource percent* is greater than zero as that would indicate [underutilized workload isolation](#underutilized-workload-isolation).|Min, Avg, Max |
29+
|Workload group allocation by system percent | This metric displays the percentage allocation of resources relative to the entire system.<br><br>Consider a workload group `DataLoads` with a `REQUEST_MIN_RESOURCE_GRANT_PERCENT` configured at 25%. *Workload group allocation by system percent* value filtered to `DataLoads` would be 25% (25% / 100%) if a single query were running in this workload group.|Min, Avg, Max |
30+
|Workload group query timeouts |Queries for the workload group that have timed out. Query timeouts reported by this metric are only once the query has started executing (it does not include wait time due to locking or resource waits).<br><br>Query timeout is configured using the `QUERY_EXECUTION_TIMEOUT_SEC` parameter in the [CREATE WORKLOAD GROUP](https://docs.microsoft.com/sql/t-sql/statements/create-workload-group-transact-sql?view=azure-sqldw-latest) syntax. Increasing the value could reduce the number of query timeouts.<br><br>Consider increasing the `REQUEST_MIN_RESOURCE_GRANT_PERCENT` parameter for the workload group to reduce the amount of timeouts and allocate more resources per query. Note, increasing `REQUEST_MIN_RESOURCE_GRANT_PERCENT` reduces the amount of concurrency for the workload group. |Sum |
31+
|Workload group queued queries | Queries for the workload group that are currently queued waiting to start execution. Queries can be queue because they are waiting for resources or locks.<br><br>Queries could be waiting for numerous reasons. If the system is overloaded and the concurrency demand is greater than what is available, queries will queue.<br><br>Consider adding more resources to the workload group by increasing the `CAP_PERCENTAGE_RESOURCE` parameter in the [CREATE WORKLOAD GROUP](https://docs.microsoft.com/sql/t-sql/statements/create-workload-group-transact-sql?view=azure-sqldw-latest) statement. If `CAP_PERCENTAGE_RESOURCE` is greater than the *Effective cap resource percent* metric, the configured workload isolation for other workload group is impacting the resources allocated to this workload group. Consider lowering `MIN_PERCENTAGE_RESOURCE` of other workload groups or scale up the instance to add more resources. |Sum |
32+
33+
## Monitoring scenarios and actions
34+
Below are a series of chart configurations to highlight workload management metric usage for troubleshooting along with associated actions to address the issue.
35+
36+
### Underutilized workload isolation
37+
Consider the following workload group and classifier configuration where a workload group named `wgPriority` is created and *TheCEO* `membername` is mapped to it using the `wcCEOPriority` workload classifier. The `wgPriority` workload group has 25% workload isolation configured for it (`MIN_PERCENTAGE_RESOURCE` = 25). Each query submitted by *TheCEO* is given 5% of system resources (`REQUEST_MIN_RESOURCE_GRANT_PERCENT` = 5).
38+
```sql
39+
CREATE WORKLOAD GROUP wgPriority
40+
WITH ( MIN_PERCENTAGE_RESOURCE = 25
41+
,CAP_PERCENTAGE_RESOURCE = 50
42+
,REQUEST_MIN_RESOURCE_GRANT_PERCENT = 5);
43+
44+
CREATE WORKLOAD CLASSIFIER wcCEOPriority
45+
WITH ( WORKLOAD_GROUP = 'wgPriority'
46+
,MEMBERNAME = 'TheCEO');
47+
```
48+
The below chart is configured as follows:
49+
Metric 1: *Effective min resource percent* (Avg aggregation, `blue line`)
50+
Metric 2: *Workload group allocation by system percent* (Avg aggregation, `purple line`)
51+
Filter: [Workload Group] = `wgPriority`
52+
![underutilized-wg.png](media/sql-data-warehouse-workload-management-portal-monitor/underutilized-wg.png)
53+
The chart shows that with 25% workload isolation, only 10% is being used on average. In this case, the `MIN_PERCENTAGE_RESOURCE` parameter value could be lowered to between 10 or 15 and allow for other workloads on the system to consume the resources.
54+
55+
### Workload group bottleneck
56+
Consider the following workload group and classifier configuration where a workload group named `wgDataAnalyst` is created and the *DataAnalyst* `membername` is mapped to it using the `wcDataAnalyst` workload classifier. The `wgDataAnalyst` workload group has 6% workload isolation configured for it (`MIN_PERCENTAGE_RESOURCE` = 6) and a resource limit of 9% (`CAP_PERCENTAGE_RESOURCE` = 9). Each query submitted by the *DataAnalyst* is given 3% of system resources (`REQUEST_MIN_RESOURCE_GRANT_PERCENT` = 3).
57+
58+
```sql
59+
CREATE WORKLOAD GROUP wgDataAnalyst
60+
WITH ( MIN_PERCENTAGE_RESOURCE = 6
61+
,CAP_PERCENTAGE_RESOURCE = 9
62+
,REQUEST_MIN_RESOURCE_GRANT_PERCENT = 3);
63+
64+
CREATE WORKLOAD CLASSIFIER wcDataAnalyst
65+
WITH ( WORKLOAD_GROUP = 'wgDataAnalyst'
66+
,MEMBERNAME = 'DataAnalyst');
67+
```
68+
The below chart is configured as follows:
69+
Metric 1: *Effective cap resource percent* (Avg aggregation, `blue line`)
70+
Metric 2: *Workload group allocation by max resource percent* (Avg aggregation, `purple line`)
71+
Metric 3: *Workload group queued queries* (Sum aggregation, `turquoise line`)
72+
Filter: [Workload Group] = `wgDataAnalyst`
73+
![bottle-necked-wg](media/sql-data-warehouse-workload-management-portal-monitor/bottle-necked-wg.png)
74+
The chart shows that with a 9% cap on resources, the workload group is 90%+ utilized (from the *Workload group allocation by max resource percent metric*). There is a steady queuing of queries as shown from the *Workload group queued queries metric*. In this case, increasing the `CAP_PERCENTAGE_RESOURCE` to a value higher than 9% will allow more queries to execute concurrently. Increasing the `CAP_PERCENTAGE_RESOURCE` assumes that there are enough resources available and not isolated by other workload groups. Verify the cap increased by checking the *Effective cap resource percent metric*. If more throughput is desired, also consider increasing the `REQUEST_MIN_RESOURCE_GRANT_PERCENT` to a value greater than 3. Increasing the `REQUEST_MIN_RESOURCE_GRANT_PERCENT` could allow queries to run faster.
75+
76+
## Next steps
77+
[Quickstart: Configure workload isolation using T-SQL](quickstart-configure-workload-isolation-tsql.md)<br>
78+
[CREATE WORKLOAD GROUP (Transact-SQL)](https://docs.microsoft.com/sql/t-sql/statements/create-workload-group-transact-sql?view=azure-sqldw-latest)<br>
79+
[CREATE WORKLOAD CLASSIFIER (Transact-SQL)](https://docs.microsoft.com/sql/t-sql/statements/create-workload-classifier-transact-sql?view=azure-sqldw-latest)<br>
80+
[Monitoring resource utilization](sql-data-warehouse-concept-resource-utilization-query-activity.md)
81+

articles/sql-data-warehouse/sql-data-warehouse-workload-management.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ manager: craigg
77
ms.service: sql-data-warehouse
88
ms.topic: conceptual
99
ms.subservice: workload-management
10-
ms.date: 10/30/2019
10+
ms.date: 01/13/2020
1111
ms.author: rortloff
1212
ms.reviewer: jrasnick
1313
ms.custom: seo-lt-2019
@@ -48,4 +48,5 @@ Workload isolation reserves resources for a workload group. Resources reserved
4848

4949
- For more information about workload classification, see [Workload Classification](sql-data-warehouse-workload-classification.md).
5050
- For more information about workload isolation, see [Workload Isolation](sql-data-warehouse-workload-isolation.md).
51-
- For more information about workload importance, see [Workload Importance](sql-data-warehouse-workload-importance.md).
51+
- For more information about workload importance, see [Workload Importance](sql-data-warehouse-workload-importance.md).
52+
- For more information about workload management monitoring, see [Workload Management Portal Monitoring](sql-data-warehouse-workload-management-portal-monitor.md).

0 commit comments

Comments
 (0)