You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/synapse-analytics/sql-data-warehouse/resource-classes-for-workload-management.md
+35-28Lines changed: 35 additions & 28 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,20 +4,21 @@ description: Guidance for using resource classes to manage concurrency and compu
4
4
author: WilliamDAssafMSFT
5
5
ms.author: wiassaf
6
6
ms.reviewer: whhender
7
-
ms.date: 02/04/2020
7
+
ms.date: 07/29/2024
8
8
ms.service: synapse-analytics
9
9
ms.subservice: sql-dw
10
10
ms.topic: conceptual
11
-
ms.custom: azure-synapse
11
+
ms.custom:
12
+
- azure-synapse
12
13
---
13
14
14
15
# Workload management with resource classes in Azure Synapse Analytics
15
16
16
-
Guidance for using resource classes to manage memory and concurrency for Synapse SQL pool queries in Azure Synapse.
17
+
Guidance for using resource classes to manage memory and concurrency for Synapse SQL pool queries in Azure Synapse.
17
18
18
19
## What are resource classes
19
20
20
-
The performance capacity of a query is determined by the user's resource class. Resource classes are pre-determined resource limits in Synapse SQL pool that govern compute resources and concurrency for query execution. Resource classes can help you configure resources for your queries by setting limits on the number of queries that run concurrently and on the compute-resources assigned to each query. There's a trade-off between memory and concurrency.
21
+
The performance capacity of a query is determined by the user's resource class. Resource classes are pre-determined resource limits in Synapse SQL pool that govern compute resources and concurrency for query execution. Resource classes can help you configure resources for your queries by setting limits on the number of queries that run concurrently and on the compute-resources assigned to each query. There's a trade-off between memory and concurrency.
21
22
22
23
- Smaller resource classes reduce the maximum memory per query, but increase concurrency.
23
24
- Larger resource classes increase the maximum memory per query, but reduce concurrency.
@@ -27,14 +28,14 @@ There are two types of resource classes:
27
28
- Static resources classes, which are well suited for increased concurrency on a data set size that is fixed.
28
29
- Dynamic resource classes, which are well suited for data sets that are growing in size and need increased performance as the service level is scaled up.
29
30
30
-
Resource classes use concurrency slots to measure resource consumption. [Concurrency slots](#concurrency-slots) are explained later in this article.
31
+
Resource classes use concurrency slots to measure resource consumption. [Concurrency slots](#concurrency-slots) are explained later in this article.
31
32
32
33
- To view the resource utilization for the resource classes, see [Memory and concurrency limits](memory-concurrency-limits.md).
33
34
- To adjust the resource class, you can run the query under a different user or [change the current user's resource class](#change-a-users-resource-class) membership.
34
35
35
36
### Static resource classes
36
37
37
-
Static resource classes allocate the same amount of memory regardless of the current performance level, which is measured in [data warehouse units](what-is-a-data-warehouse-unit-dwu-cdwu.md). Since queries get the same memory allocation regardless of the performance level, [scaling out the data warehouse](quickstart-scale-compute-portal.md) allows more queries to run within a resource class. Static resource classes are ideal if the data volume is known and constant.
38
+
Static resource classes allocate the same amount of memory regardless of the current performance level, which is measured in [data warehouse units](what-is-a-data-warehouse-unit-dwu-cdwu.md). Since queries get the same memory allocation regardless of the performance level, [scaling out the data warehouse](quickstart-scale-compute-portal.md) allows more queries to run within a resource class. Static resource classes are ideal if the data volume is known and constant.
38
39
39
40
The static resource classes are implemented with these pre-defined database roles:
40
41
@@ -49,14 +50,16 @@ The static resource classes are implemented with these pre-defined database role
49
50
50
51
### Dynamic resource classes
51
52
52
-
Dynamic Resource Classes allocate a variable amount of memory depending on the current service level. While static resource classes are beneficial for higher concurrency and static data volumes, dynamic resource classes are better suited for a growing or variable amount of data. When you scale up to a larger service level, your queries automatically get more memory.
53
+
Dynamic resource classes allocate a variable amount of memory depending on the current service level. While static resource classes are beneficial for higher concurrency and static data volumes, dynamic resource classes are better suited for a growing or variable amount of data. When you scale up to a larger service level, your queries automatically get more memory.
53
54
54
-
Except smallrc, the dynamic resource classes are implemented with these pre-defined database roles:
55
+
Except for smallrc, the dynamic resource classes are implemented with these pre-defined database roles:
55
56
56
57
- mediumrc
57
58
- largerc
58
59
- xlargerc
59
60
61
+
Smallrc does not appear as a database role, but is the [Default resource class](#default-resource-class).
62
+
60
63
The memory allocation for each resource class is as follows.
@@ -72,7 +75,7 @@ The memory allocation for each resource class is as follows.
72
75
73
76
By default, each user is a member of the dynamic resource class **smallrc**.
74
77
75
-
The resource class of the service administrator is fixed at smallrc and cannot be changed. The service administrator is the user created during the provisioning process. The service administrator in this context is the login specified for the "Server admin login" when creating a new Synapse SQL pool with a new server.
78
+
The resource class of the service administrator is fixed at smallrc and cannot be changed. The service administrator is the user created during the provisioning process. The service administrator in this context is the login specified for the "Server admin login" when creating a new Synapse SQL pool with a new server.
76
79
77
80
> [!NOTE]
78
81
> Users or groups defined as Active Directory admin are also service administrators.
@@ -100,7 +103,7 @@ These operations are governed by resource classes:
100
103
101
104
### Operations not governed by resource classes
102
105
103
-
Some queries always run in the smallrc resource class even though the user is a member of a larger resource class. These exempt queries do not count towards the concurrency limit. For example, if the concurrency limit is 16, many users can be selecting from system views without impacting the available concurrency slots.
106
+
Some queries always run in the smallrc resource class even though the user is a member of a larger resource class. These exempt queries do not count toward the concurrency limit. For example, if the concurrency limit is 16, many users can be selecting from system views without affecting the available concurrency slots.
104
107
105
108
The following statements are exempt from resource classes and always run in smallrc:
106
109
@@ -129,7 +132,7 @@ Removed as these two are not confirmed / supported under Azure Synapse Analytics
129
132
130
133
## Concurrency slots
131
134
132
-
Concurrency slots are a convenient way to track the resources available for query execution. They are like tickets that you purchase to reserve seats at a concert because seating is limited. The total number of concurrency slots per data warehouse is determined by the service level. Before a query can start executing, it must be able to reserve enough concurrency slots. When a query completes, it releases its concurrency slots.
135
+
Concurrency slots are a convenient way to track the resources available for query execution. They are like tickets that you purchase to reserve seats at a concert because seating is limited. The total number of concurrency slots per data warehouse is determined by the service level. Before a query can start executing, it must be able to reserve enough concurrency slots. When a query completes, it releases its concurrency slots.
133
136
134
137
- A query running with 10 concurrency slots can access 5 times more compute resources than a query running with 2 concurrency slots.
135
138
- If each query requires 10 concurrency slots and there are 40 concurrency slots, then only 4 queries can run concurrently.
@@ -138,7 +141,7 @@ Only resource governed queries consume concurrency slots. System queries and som
138
141
139
142
## View the resource classes
140
143
141
-
Resource classes are implemented as pre-defined database roles. There are two types of resource classes: dynamic and static. To view a list of the resource classes, use the following query:
144
+
Resource classes are implemented as predefined database roles. There are two types of resource classes: dynamic and static. To view a list of the resource classes, use the following query:
142
145
143
146
```sql
144
147
SELECT name
@@ -150,13 +153,13 @@ WHERE name LIKE '%rc%' AND type_desc = 'DATABASE_ROLE';
150
153
151
154
Resource classes are implemented by assigning users to database roles. When a user runs a query, the query runs with the user's resource class. For example, if a user is a member of the staticrc10 database role, their queries run with small amounts of memory. If a database user is a member of the xlargerc or staticrc80 database roles, their queries run with large amounts of memory.
152
155
153
-
To increase a user's resource class, use [sp_addrolemember](/sql/relational-databases/system-stored-procedures/sp-addrolemember-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true) to add the user to a database role of a large resource class. The below code adds a user to the largerc database role. Each request gets 22% of the system memory.
156
+
To increase a user's resource class, use [sp_addrolemember](/sql/relational-databases/system-stored-procedures/sp-addrolemember-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true) to add the user to a database role of a large resource class. The following code adds a user to the largerc database role. Each request gets 22% of the system memory.
154
157
155
158
```sql
156
159
EXEC sp_addrolemember 'largerc', 'loaduser';
157
160
```
158
161
159
-
To decrease the resource class, use [sp_droprolemember](/sql/relational-databases/system-stored-procedures/sp-droprolemember-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true). If 'loaduser' is not a member or any other resource classes, they go into the default smallrc resource class with a 3% memory grant.
162
+
To decrease the resource class, use [sp_droprolemember](/sql/relational-databases/system-stored-procedures/sp-droprolemember-transact-sql?toc=/azure/synapse-analytics/sql-data-warehouse/toc.json&bc=/azure/synapse-analytics/sql-data-warehouse/breadcrumb/toc.json&view=azure-sqldw-latest&preserve-view=true). If 'loaduser' is not a member or any other resource classes, they go into the default smallrc resource class with a 3% memory grant.
160
163
161
164
```sql
162
165
EXEC sp_droprolemember 'largerc', 'loaduser';
@@ -172,7 +175,7 @@ Users can be members of multiple resource classes. When a user belongs to more t
172
175
## Recommendations
173
176
174
177
>[!NOTE]
175
-
>Consider leveraging workload management capabilities ([workload isolation](sql-data-warehouse-workload-isolation.md), [classification](sql-data-warehouse-workload-classification.md) and [importance](sql-data-warehouse-workload-importance.md)) for more control over your workload and predictable performance.
178
+
>Consider leveraging workload management capabilities ([workload isolation](sql-data-warehouse-workload-isolation.md), [classification](sql-data-warehouse-workload-classification.md) and [importance](sql-data-warehouse-workload-importance.md)) for more control over your workload and predictable performance.
176
179
177
180
We recommend creating a user that is dedicated to running a specific type of query or load operation. Give that user a permanent resource class instead of changing the resource class on a frequent basis. Static resource classes afford greater overall control on the workload, so we suggest using static resource classes before considering dynamic resource classes.
178
181
@@ -189,9 +192,9 @@ Once you have determined the memory requirement, choose whether to assign the lo
189
192
190
193
### Resource classes for queries
191
194
192
-
Some queries are compute-intensive and some aren't.
195
+
Some queries are compute-intensive and some aren't.
193
196
194
-
- Choose a dynamic resource class when queries are complex, but don't need high concurrency. For example, generating daily or weekly reports is an occasional need for resources. If the reports are processing large amounts of data, scaling the data warehouse provides more memory to the user's existing resource class.
197
+
- Choose a dynamic resource class when queries are complex, but don't need high concurrency. For example, generating daily or weekly reports is an occasional need for resources. If the reports are processing large amounts of data, scaling the data warehouse provides more memory to the user's existing resource class.
195
198
- Choose a static resource class when resource expectations vary throughout the day. For example, a static resource class works well when the data warehouse is queried by many people. When scaling the data warehouse, the amount of memory allocated to the user doesn't change. Consequently, more queries can be executed in parallel on the system.
196
199
197
200
Proper memory grants depend on many factors, such as the amount of data queried, the nature of the table schemas, and various joins, select, and group predicates. In general, allocating more memory allows queries to complete faster, but reduces the overall concurrency. If concurrency is not an issue, over-allocating memory does not harm throughput.
@@ -200,21 +203,25 @@ To tune performance, use different resource classes. The next section gives a st
200
203
201
204
## Example code for finding the best resource class
202
205
203
-
You can use the following specified stored procedure to figure out concurrency and memory grant per resource class at a given SLO and the best resource class for memory intensive CCI operations on non-partitioned CCI table at a given resource class:
206
+
Use the `prc_workload_management_by_DWU` stored procedure to:
207
+
208
+
209
+
- See the concurrency and memory grant per resource class at a given SLO.
210
+
- Provide `NULL` for both schema and tablename.
211
+
- See the best resource class for the memory-intensive CCI operations (load, copy table, rebuild index, etc.) on nonpartitioned CCI table at a given resource class.
212
+
- The stored proc uses table schema to find out the required memory grant.
204
213
205
-
Here's the purpose of this stored procedure:
214
+
For examples, see [Usage example](#usage-example).
206
215
207
-
1. To see the concurrency and memory grant per resource class at a given SLO. User needs to provide NULL for both schema and tablename as shown in this example.
208
-
2. To see the best resource class for the memory-intensive CCI operations (load, copy table, rebuild index, etc.) on non-partitioned CCI table at a given resource class. The stored proc uses table schema to find out the required memory grant.
209
216
210
217
### Dependencies & Restrictions
211
218
212
219
- This stored procedure isn't designed to calculate the memory requirement for a partitioned cci table.
213
220
- This stored procedure doesn't take memory requirements into account for the SELECT part of CTAS/INSERT-SELECT and assumes it's a SELECT.
214
221
- This stored procedure uses a temp table, which is available in the session where this stored procedure was created.
215
-
- This stored procedure depends on the current offerings (for example, hardware configuration, DMS config), and if any of that changes then this stored proc won't work correctly.
216
-
- This stored procedure depends on existing concurrency limit offerings and if these change then this stored procedure won't work correctly.
217
-
- This stored procedure depends on existing resource class offerings and if these change then this stored procedure won't work correctly.
222
+
- This stored procedure depends on the current offerings (for example, hardware configuration, DMS config), and if any of that changes then this stored proc won't work correctly.
223
+
- This stored procedure depends on existing concurrency limit offerings and if these change then this stored procedure won't work correctly.
224
+
- This stored procedure depends on existing resource class offerings and if these change then this stored procedure won't work correctly.
218
225
219
226
>[!NOTE]
220
227
>If you are not getting output after executing stored procedure with parameters provided, then there could be two cases.
The following statement creates Table1 that is used in the preceding examples.
252
+
The following statement creates `Table1` that is used in the preceding examples.
246
253
`CREATE TABLE Table1 (a int, b varchar(50), c decimal (18,10), d char(10), e varbinary(15), f float, g datetime, h date);`
247
254
248
255
### Stored procedure definition
@@ -576,6 +583,6 @@ SELECT CASE
576
583
GO
577
584
```
578
585
579
-
## Next steps
586
+
## Related content
580
587
581
588
For more information about managing database users and security, see [Secure a database in Synapse SQL](sql-data-warehouse-overview-manage-security.md). For more information about how larger resource classes can improve clustered columnstore index quality, see [Memory optimizations for columnstore compression](sql-data-warehouse-memory-optimizations-for-columnstore-compression.md).
0 commit comments