You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/synapse-analytics/sql-analytics/best-practices-sql-on-demand.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -63,4 +63,4 @@ As CETAS generates Parquet files, statistics will be automatically created when
63
63
64
64
## Next steps
65
65
66
-
Review the [Troubleshooting](../../sql-data-warehouse/sql-data-warehouse-troubleshoot.md) article for common issues and solutions. If you're working with SQL pools rather than SQL on-demand, please see the [Best Practices for SQL pools](best-practices-sql-pool.md) article for specific guidance.
66
+
Review the [Troubleshooting](../sql-data-warehouse/sql-data-warehouse-troubleshoot.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json) article for common issues and solutions. If you're working with SQL pools rather than SQL on-demand, please see the [Best Practices for SQL pools](best-practices-sql-pool.md) article for specific guidance.
Copy file name to clipboardExpand all lines: articles/synapse-analytics/sql-analytics/best-practices-sql-pool.md
+15-15Lines changed: 15 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,7 +19,7 @@ This article provides a collection of best practices to help you achieve optimal
19
19
For SQL pools loading guidance, see [Guidance for loading data](data-loading-best-practices.md).
20
20
21
21
## Reduce cost with pause and scale
22
-
For more information about reducing costs through pausing and scaling, see [Manage compute](../../sql-data-warehouse/sql-data-warehouse-manage-compute-overview.md).
22
+
For more information about reducing costs through pausing and scaling, see [Manage compute](../sql-data-warehouse/sql-data-warehouse-manage-compute-overview.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json).
23
23
24
24
## Maintain statistics
25
25
While SQL Server automatically detects and creates or updates statistics on columns, SQL pools require manual maintenance of statistics. You'll want to maintain your statistics to ensure that the SQL pool plans are optimized. The plans created by the optimizer are only as good as the available statistics.
@@ -52,10 +52,10 @@ To maximize throughput when using Gzip text files, break up files into 60 or mor
52
52
-[Load data](data-loading-overview.md)
53
53
-[Guide for using PolyBase](data-loading-best-practices.md)
54
54
-[Azure SQL pool loading patterns and strategies](https://blogs.msdn.microsoft.com/sqlcat/20../../azure-sql-data-warehouse-loading-patterns-and-strategies/)
55
-
-[Load Data with Azure Data Factory](../../data-factory/load-azure-sql-data-warehouse.md)
56
-
-[Move data with Azure Data Factory](../../data-factory/transform-data-using-machine-learning.md)
55
+
-[Load Data with Azure Data Factory](../../data-factory/load-azure-sql-data-warehouse.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json)
56
+
-[Move data with Azure Data Factory](../../data-factory/transform-data-using-machine-learning.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json)
-[Create table as select (CTAS)](../../sql-data-warehouse/sql-data-warehouse-develop-ctas.md)
58
+
-[Create table as select (CTAS)](../sql-data-warehouse/sql-data-warehouse-develop-ctas.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json)
59
59
60
60
## Load then query external tables
61
61
Polybase isn't optimal for queries. Polybase tables for SQL pools currently only support Azure blob files and Azure Data Lake storage. These files don't have any compute resources backing them. As a result, SQL pools can't offload this work and must read the entire file by loading it to tempdb so it can read the data.
@@ -74,7 +74,7 @@ When loading a distributed table, your incoming data shouldn't be sorted on the
74
74
75
75
The article links provided below will give you additional details about improving performance via selecting a distribution column. Also, you'll find information about how to define a distributed table in the WITH clause of your CREATE TABLE statement:
-[CREATE TABLE AS SELECT](https://msdn.microsoft.com/library/mt204041.aspx)
@@ -86,7 +86,7 @@ Having too many partitions can reduce the effectiveness of clustered columnstore
86
86
87
87
One option to consider is using a granularity that is lower than what you've implemented using SQL Server. For example, consider using weekly or monthly partitions instead of daily partitions.
88
88
89
-
More information about partitioning is detailed in the [Table partitioning](../../sql-data-warehouse/sql-data-warehouse-tables-partition.md) article.
89
+
More information about partitioning is detailed in the [Table partitioning](../sql-data-warehouse/sql-data-warehouse-tables-partition.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json) article.
90
90
91
91
## Minimize transaction sizes
92
92
INSERT, UPDATE, and DELETE statements run in a transaction. When they fail, they must be rolled back. To reduce the potential for a long rollback, minimize transaction sizes whenever possible. Minimizing transaction sizes can be done by dividing INSERT, UPDATE, and DELETE statements into parts. For example, if you have an INSERT that you expect to take 1 hour, you can break up the INSERT into four parts. Each run will then be shortened to 15 minutes.
@@ -99,10 +99,10 @@ Another way to eliminate rollbacks is to use Metadata Only operations like parti
99
99
For unpartitioned tables, consider using a CTAS to write the data you want to keep in a table rather than using DELETE. If a CTAS takes the same amount of time, it's much safer to run since it has minimal transaction logging and can be canceled quickly if needed.
100
100
101
101
Further information on content related to this section is included in the articles below:
102
-
-[Create table as select (CTAS)](../../sql-data-warehouse/sql-data-warehouse-develop-ctas.md)
102
+
-[Create table as select (CTAS)](../sql-data-warehouse/sql-data-warehouse-develop-ctas.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json)
@@ -125,7 +125,7 @@ For additional guidance, refer to the [Temporary tables](https://msdn.microsoft.
125
125
126
126
Clustered columnstore indexes are one of the most efficient ways you can store your data in SQL pool. By default, tables in SQL pool are created as Clustered ColumnStore. To get the best performance for queries on columnstore tables, having good segment quality is important. When rows are written to columnstore tables under memory pressure, columnstore segment quality may suffer.
127
127
128
-
Segment quality can be measured by the number of rows in a compressed Row Group. See the [Causes of poor columnstore index quality](../../sql-data-warehouse/sql-data-warehouse-tables-index.md#causes-of-poor-columnstore-index-quality) in the [Table indexes](../../sql-data-warehouse/sql-data-warehouse-tables-index.md) article for step-by-step instructions on detecting and improving segment quality for clustered columnstore tables.
128
+
Segment quality can be measured by the number of rows in a compressed Row Group. See the [Causes of poor columnstore index quality](../sql-data-warehouse/sql-data-warehouse-tables-index.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json#causes-of-poor-columnstore-index-quality) in the [Table indexes](../sql-data-warehouse/sql-data-warehouse-tables-index.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json) article for step-by-step instructions on detecting and improving segment quality for clustered columnstore tables.
129
129
130
130
Because high-quality columnstore segments are important, it's a good idea to use users IDs that are in the medium or large resource class for loading data. Using lower [data warehouse units](resource-consumption-models.md) means you want to assign a larger resource class to your loading user.
131
131
@@ -138,23 +138,23 @@ If you partition your data, each partition will need to have 1 million rows to b
138
138
139
139
If your table doesn't have 6 billion rows, you have two main options. Either reduce the number of partitions or consider using a heap table instead. It also may be worth experimenting to see if better performance can be gained by using a heap table with secondary indexes rather than a columnstore table.
140
140
141
-
When querying a columnstore table, queries will run faster if you select only the columns you need. More information on table and columnstore indexes and can be found within the [Table indexes](../../sql-data-warehouse/sql-data-warehouse-tables-index.md), [Columnstore indexes guide](https://msdn.microsoft.com/library/gg492088.aspx), and [Rebuilding columnstore indexes](../../sql-data-warehouse/sql-data-warehouse-tables-index.md#rebuilding-indexes-to-improve-segment-quality) articles.
141
+
When querying a columnstore table, queries will run faster if you select only the columns you need. More information on table and columnstore indexes and can be found within the [Table indexes](../sql-data-warehouse/sql-data-warehouse-tables-index.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json), [Columnstore indexes guide](https://msdn.microsoft.com/library/gg492088.aspx), and [Rebuilding columnstore indexes](../sql-data-warehouse/sql-data-warehouse-tables-index.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json#rebuilding-indexes-to-improve-segment-quality) articles.
142
142
143
143
## Use larger resource class to improve query performance
144
144
SQL pools use resource groups as a way to allocate memory to queries. Initially, all users are assigned to the small resource class, which grants 100 MB of memory per distribution. There are always 60 distributions. Each distribution is given a minimum of 100 MB. The total system-wide memory allocation is 6,000 MB, or just under 6 GB.
145
145
146
146
Certain queries, like large joins or loads to clustered columnstore tables, will benefit from larger memory allocations. Some queries, such as pure scans, will see no benefit. Utilizing larger resource classes impacts concurrency. So, you'll want to keep these facts in mind before moving all of your users to a large resource class.
147
147
148
-
For additional information on resource classes, refer to the [Resource classes for workload management](../../sql-data-warehouse/resource-classes-for-workload-management.md) article.
148
+
For additional information on resource classes, refer to the [Resource classes for workload management](../sql-data-warehouse/resource-classes-for-workload-management.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json) article.
149
149
150
150
## Use smaller resource class to increase concurrency
151
151
If you notice a long delay in user queries, your users might be running in larger resource classes. This scenario promotes the consumption of concurrency slots, which can cause other queries to queue up. To determine if users queries are queued, run `SELECT * FROM sys.dm_pdw_waits` to see if any rows are returned.
152
152
153
-
The [Resource classes for workload management](../../sql-data-warehouse/resource-classes-for-workload-management.md) and [sys.dm_pdw_waits](https://msdn.microsoft.com/library/mt203893.aspx) articles will provide you with more information.
153
+
The [Resource classes for workload management](../sql-data-warehouse/resource-classes-for-workload-management.md) and [sys.dm_pdw_waits](https://msdn.microsoft.com/library/mt203893.aspx) articles will provide you with more information.
154
154
155
155
## Use DMVs to monitor and optimize your queries
156
156
SQL pools have several DMVs that can be used to monitor query execution. The monitoring article below walks you through step-by-step instructions on how to view details of an executing query. To quickly find queries in these DMVs, using the LABEL option with your queries can help. For additional detailed information, please see the articles included in the list below:
157
-
-[Monitor your workload using DMVs](../../sql-data-warehouse/sql-data-warehouse-manage-monitor.md)
157
+
-[Monitor your workload using DMVs](../sql-data-warehouse/sql-data-warehouse-manage-monitor.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json)
@@ -167,7 +167,7 @@ SQL pools have several DMVs that can be used to monitor query execution. The mo
167
167
168
168
## Next steps
169
169
170
-
Also see the [Troubleshooting](../../sql-data-warehouse/sql-data-warehouse-troubleshoot.md) article for common issues and solutions.
170
+
Also see the [Troubleshooting](../sql-data-warehouse/sql-data-warehouse-troubleshoot.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json) article for common issues and solutions.
171
171
172
172
If you need information not provided in this article, use the "Search for docs" on the left side of this page to search all of the SQL pool documents. The [SQL pool Forum](https://social.msdn.microsoft.com/Forums/sqlserver/home?forum=AzureSQLDataWarehouse) is a place for you to pose questions to other users and to the SQL pool Product Group.
Copy file name to clipboardExpand all lines: articles/synapse-analytics/sql-analytics/connect-overview.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -83,6 +83,6 @@ For executing **SQL on-demand** queries, recommended tools are [Azure Data Studi
83
83
84
84
85
85
## Next steps
86
-
To connect and query with Visual Studio, see [Query with Visual Studio](../../sql-data-warehouse/sql-data-warehouse-query-visual-studio.md). To learn more about authentication options, see [Authentication to SQL Analytics](../../sql-data-warehouse/sql-data-warehouse-authentication.md).
86
+
To connect and query with Visual Studio, see [Query with Visual Studio](../sql-data-warehouse/sql-data-warehouse-query-visual-studio.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json). To learn more about authentication options, see [Authentication to SQL Analytics](../sql-data-warehouse/sql-data-warehouse-authentication.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json).
For executing **SQL on-demand** queries, recommended tools are [Azure Data Studio](get-started-azure-data-studio.md) and Azure Synapse Studio.
48
48
49
49
## Next steps
50
-
To start querying your analytics with Visual Studio and other applications, see [Query with Visual Studio](../../sql-data-warehouse/sql-data-warehouse-query-visual-studio.md).
50
+
To start querying your analytics with Visual Studio and other applications, see [Query with Visual Studio](../sql-data-warehouse/sql-data-warehouse-query-visual-studio.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json).
Copy file name to clipboardExpand all lines: articles/synapse-analytics/sql-analytics/data-load-columnstore-compression.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -135,10 +135,10 @@ OPTION (MAXDOP 1);
135
135
136
136
DWU size and the user resource class together determine how much memory is available for a user query. To increase the memory grant for a load query, you can either increase the number of DWUs or increase the resource class.
137
137
138
-
- To increase the DWUs, see [How do I scale performance?]../synapse-analytics/sql-data-warehouse/quickstart-scale-compute-portal.md)
139
-
- To change the resource class for a query, see [Change a user resource class example](../../synapse-analytics/sql-data-warehouse/resource-classes-for-workload-management.md#change-a-users-resource-class).
138
+
- To increase the DWUs, see [How do I scale performance?](../sql-data-warehouse/quickstart-scale-compute-portal.md)
139
+
- To change the resource class for a query, see [Change a user resource class example](../sql-data-warehouse/resource-classes-for-workload-management.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json#change-a-users-resource-class).
140
140
141
141
## Next steps
142
142
143
-
To find more ways to improve performance in SQL Analytics, see the [Performance overview](../../synapse-analytics/sql-data-warehouse/cheat-sheet.md).
143
+
To find more ways to improve performance in SQL Analytics, see the [Performance overview](../sql-data-warehouse/cheat-sheet.md?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json).
0 commit comments