Skip to content

Commit 26f4c27

Browse files
julieMSFTjulieMSFT
committed
move files and redirection
1 parent 047fe59 commit 26f4c27

File tree

62 files changed

+2140
-2092
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

62 files changed

+2140
-2092
lines changed

‎articles/data-factory/connector-azure-sql-data-warehouse.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -735,7 +735,7 @@ Settings specific to Azure Synapse Analytics are available in the **Settings** t
735735
When you copy data from or to Azure Synapse Analytics, the following mappings are used from Azure Synapse Analytics data types to Azure Data Factory interim data types. See [schema and data type mappings](copy-activity-schema-and-type-mapping.md) to learn how Copy Activity maps the source schema and data type to the sink.
736736

737737
>[!TIP]
738-
>Refer to [Table data types in Azure Synapse Analytics](../sql-data-warehouse/sql-data-warehouse-tables-data-types.md) article on SQL DW supported data types and the workarounds for unsupported ones.
738+
>Refer to [Table data types in Azure Synapse Analytics](../synapse-analytics/sql-data-warehouse/sql-data-warehouse-tables-data-types.md) article on SQL DW supported data types and the workarounds for unsupported ones.
739739
740740
| Azure Synapse Analytics data type | Data Factory interim data type |
741741
| :------------------------------------ | :----------------------------- |

‎articles/sql-data-warehouse/sql-data-warehouse-develop-transactions.md

Lines changed: 2 additions & 197 deletions
Original file line numberDiff line numberDiff line change
@@ -1,199 +1,4 @@
11
---
2-
title: Using transactions
3-
description: Tips for implementing transactions in Azure SQL Data Warehouse for developing solutions.
4-
services: sql-data-warehouse
5-
author: XiaoyuMSFT
6-
manager: craigg
7-
ms.service: sql-data-warehouse
8-
ms.topic: conceptual
9-
ms.subservice: development
10-
ms.date: 03/22/2019
11-
ms.author: xiaoyul
12-
ms.reviewer: igorstan
13-
ms.custom: seo-lt-2019
2+
redirect_url: /azure/synapse-analytics/sql-data-warehouse/sql-data-warehouse-develop-transactions
3+
redirect_document_id: true
144
---
15-
16-
# Using transactions in SQL Data Warehouse
17-
Tips for implementing transactions in Azure SQL Data Warehouse for developing solutions.
18-
19-
## What to expect
20-
As you would expect, SQL Data Warehouse supports transactions as part of the data warehouse workload. However, to ensure the performance of SQL Data Warehouse is maintained at scale some features are limited when compared to SQL Server. This article highlights the differences and lists the others.
21-
22-
## Transaction isolation levels
23-
SQL Data Warehouse implements ACID transactions. The isolation level of the transactional support is default to READ UNCOMMITTED. You can change it to READ COMMITTED SNAPSHOT ISOLATION by turning ON the READ_COMMITTED_SNAPSHOT database option for a user database when connected to the master database. Once enabled, all transactions in this database are executed under READ COMMITTED SNAPSHOT ISOLATION and setting READ UNCOMMITTED on session level will not be honored. Check [ALTER DATABASE SET options (Transact-SQL)](https://docs.microsoft.com/sql/t-sql/statements/alter-database-transact-sql-set-options?view=azure-sqldw-latest) for details.
24-
25-
## Transaction size
26-
A single data modification transaction is limited in size. The limit is applied per distribution. Therefore, the total allocation can be calculated by multiplying the limit by the distribution count. To approximate the maximum number of rows in the transaction divide the distribution cap by the total size of each row. For variable length columns, consider taking an average column length rather than using the maximum size.
27-
28-
In the table below the following assumptions have been made:
29-
30-
* An even distribution of data has occurred
31-
* The average row length is 250 bytes
32-
33-
## Gen2
34-
35-
| [DWU](../synapse-analytics/sql-data-warehouse/sql-data-warehouse-overview-what-is.md) | Cap per distribution (GB) | Number of Distributions | MAX transaction size (GB) | # Rows per distribution | Max Rows per transaction |
36-
| --- | --- | --- | --- | --- | --- |
37-
| DW100c |1 |60 |60 |4,000,000 |240,000,000 |
38-
| DW200c |1.5 |60 |90 |6,000,000 |360,000,000 |
39-
| DW300c |2.25 |60 |135 |9,000,000 |540,000,000 |
40-
| DW400c |3 |60 |180 |12,000,000 |720,000,000 |
41-
| DW500c |3.75 |60 |225 |15,000,000 |900,000,000 |
42-
| DW1000c |7.5 |60 |450 |30,000,000 |1,800,000,000 |
43-
| DW1500c |11.25 |60 |675 |45,000,000 |2,700,000,000 |
44-
| DW2000c |15 |60 |900 |60,000,000 |3,600,000,000 |
45-
| DW2500c |18.75 |60 |1125 |75,000,000 |4,500,000,000 |
46-
| DW3000c |22.5 |60 |1,350 |90,000,000 |5,400,000,000 |
47-
| DW5000c |37.5 |60 |2,250 |150,000,000 |9,000,000,000 |
48-
| DW6000c |45 |60 |2,700 |180,000,000 |10,800,000,000 |
49-
| DW7500c |56.25 |60 |3,375 |225,000,000 |13,500,000,000 |
50-
| DW10000c |75 |60 |4,500 |300,000,000 |18,000,000,000 |
51-
| DW15000c |112.5 |60 |6,750 |450,000,000 |27,000,000,000 |
52-
| DW30000c |225 |60 |13,500 |900,000,000 |54,000,000,000 |
53-
54-
## Gen1
55-
56-
| [DWU](../synapse-analytics/sql-data-warehouse/sql-data-warehouse-overview-what-is.md) | Cap per distribution (GB) | Number of Distributions | MAX transaction size (GB) | # Rows per distribution | Max Rows per transaction |
57-
| --- | --- | --- | --- | --- | --- |
58-
| DW100 |1 |60 |60 |4,000,000 |240,000,000 |
59-
| DW200 |1.5 |60 |90 |6,000,000 |360,000,000 |
60-
| DW300 |2.25 |60 |135 |9,000,000 |540,000,000 |
61-
| DW400 |3 |60 |180 |12,000,000 |720,000,000 |
62-
| DW500 |3.75 |60 |225 |15,000,000 |900,000,000 |
63-
| DW600 |4.5 |60 |270 |18,000,000 |1,080,000,000 |
64-
| DW1000 |7.5 |60 |450 |30,000,000 |1,800,000,000 |
65-
| DW1200 |9 |60 |540 |36,000,000 |2,160,000,000 |
66-
| DW1500 |11.25 |60 |675 |45,000,000 |2,700,000,000 |
67-
| DW2000 |15 |60 |900 |60,000,000 |3,600,000,000 |
68-
| DW3000 |22.5 |60 |1,350 |90,000,000 |5,400,000,000 |
69-
| DW6000 |45 |60 |2,700 |180,000,000 |10,800,000,000 |
70-
71-
The transaction size limit is applied per transaction or operation. It is not applied across all concurrent transactions. Therefore each transaction is permitted to write this amount of data to the log.
72-
73-
To optimize and minimize the amount of data written to the log, please refer to the [Transactions best practices](../synapse-analytics/sql-data-warehouse/sql-data-warehouse-develop-best-practices-transactions.md) article.
74-
75-
> [!WARNING]
76-
> The maximum transaction size can only be achieved for HASH or ROUND_ROBIN distributed tables where the spread of the data is even. If the transaction is writing data in a skewed fashion to the distributions then the limit is likely to be reached prior to the maximum transaction size.
77-
> <!--REPLICATED_TABLE-->
78-
>
79-
>
80-
81-
## Transaction state
82-
SQL Data Warehouse uses the XACT_STATE() function to report a failed transaction using the value -2. This value means the transaction has failed and is marked for rollback only.
83-
84-
> [!NOTE]
85-
> The use of -2 by the XACT_STATE function to denote a failed transaction represents different behavior to SQL Server. SQL Server uses the value -1 to represent an uncommittable transaction. SQL Server can tolerate some errors inside a transaction without it having to be marked as uncommittable. For example `SELECT 1/0` would cause an error but not force a transaction into an uncommittable state. SQL Server also permits reads in the uncommittable transaction. However, SQL Data Warehouse does not let you do this. If an error occurs inside a SQL Data Warehouse transaction it will automatically enter the -2 state and you will not be able to make any further select statements until the statement has been rolled back. It is therefore important to check that your application code to see if it uses XACT_STATE() as you may need to make code modifications.
86-
>
87-
>
88-
89-
For example, in SQL Server you might see a transaction that looks like the following:
90-
91-
```sql
92-
SET NOCOUNT ON;
93-
DECLARE @xact_state smallint = 0;
94-
95-
BEGIN TRAN
96-
BEGIN TRY
97-
DECLARE @i INT;
98-
SET @i = CONVERT(INT,'ABC');
99-
END TRY
100-
BEGIN CATCH
101-
SET @xact_state = XACT_STATE();
102-
103-
SELECT ERROR_NUMBER() AS ErrNumber
104-
, ERROR_SEVERITY() AS ErrSeverity
105-
, ERROR_STATE() AS ErrState
106-
, ERROR_PROCEDURE() AS ErrProcedure
107-
, ERROR_MESSAGE() AS ErrMessage
108-
;
109-
110-
IF @@TRANCOUNT > 0
111-
BEGIN
112-
ROLLBACK TRAN;
113-
PRINT 'ROLLBACK';
114-
END
115-
116-
END CATCH;
117-
118-
IF @@TRANCOUNT >0
119-
BEGIN
120-
PRINT 'COMMIT';
121-
COMMIT TRAN;
122-
END
123-
124-
SELECT @xact_state AS TransactionState;
125-
```
126-
127-
The preceding code gives the following error message:
128-
129-
Msg 111233, Level 16, State 1, Line 1
130-
111233; The current transaction has aborted, and any pending changes have been rolled back. Cause: A transaction in a rollback-only state was not explicitly rolled back before a DDL, DML, or SELECT statement.
131-
132-
You won't get the output of the ERROR_* functions.
133-
134-
In SQL Data Warehouse the code needs to be slightly altered:
135-
136-
```sql
137-
SET NOCOUNT ON;
138-
DECLARE @xact_state smallint = 0;
139-
140-
BEGIN TRAN
141-
BEGIN TRY
142-
DECLARE @i INT;
143-
SET @i = CONVERT(INT,'ABC');
144-
END TRY
145-
BEGIN CATCH
146-
SET @xact_state = XACT_STATE();
147-
148-
IF @@TRANCOUNT > 0
149-
BEGIN
150-
ROLLBACK TRAN;
151-
PRINT 'ROLLBACK';
152-
END
153-
154-
SELECT ERROR_NUMBER() AS ErrNumber
155-
, ERROR_SEVERITY() AS ErrSeverity
156-
, ERROR_STATE() AS ErrState
157-
, ERROR_PROCEDURE() AS ErrProcedure
158-
, ERROR_MESSAGE() AS ErrMessage
159-
;
160-
END CATCH;
161-
162-
IF @@TRANCOUNT >0
163-
BEGIN
164-
PRINT 'COMMIT';
165-
COMMIT TRAN;
166-
END
167-
168-
SELECT @xact_state AS TransactionState;
169-
```
170-
171-
The expected behavior is now observed. The error in the transaction is managed and the ERROR_* functions provide values as expected.
172-
173-
All that has changed is that the ROLLBACK of the transaction had to happen before the read of the error information in the CATCH block.
174-
175-
## Error_Line() function
176-
It is also worth noting that SQL Data Warehouse does not implement or support the ERROR_LINE() function. If you have this in your code, you need to remove it to be compliant with SQL Data Warehouse. Use query labels in your code instead to implement equivalent functionality. For more details, see the [LABEL](../synapse-analytics/sql-data-warehouse/sql-data-warehouse-develop-label.md) article.
177-
178-
## Using THROW and RAISERROR
179-
THROW is the more modern implementation for raising exceptions in SQL Data Warehouse but RAISERROR is also supported. There are a few differences that are worth paying attention to however.
180-
181-
* User-defined error messages numbers cannot be in the 100,000 - 150,000 range for THROW
182-
* RAISERROR error messages are fixed at 50,000
183-
* Use of sys.messages is not supported
184-
185-
## Limitations
186-
SQL Data Warehouse does have a few other restrictions that relate to transactions.
187-
188-
They are as follows:
189-
190-
* No distributed transactions
191-
* No nested transactions permitted
192-
* No save points allowed
193-
* No named transactions
194-
* No marked transactions
195-
* No support for DDL such as CREATE TABLE inside a user-defined transaction
196-
197-
## Next steps
198-
To learn more about optimizing transactions, see [Transactions best practices](../synapse-analytics/sql-data-warehouse/sql-data-warehouse-develop-best-practices-transactions.md). To learn about other SQL Data Warehouse best practices, see [SQL Data Warehouse best practices](../synapse-analytics/sql-data-warehouse/sql-data-warehouse-best-practices.md).
199-

‎articles/sql-data-warehouse/sql-data-warehouse-develop-user-defined-schemas.md

Lines changed: 2 additions & 119 deletions
Original file line numberDiff line numberDiff line change
@@ -1,121 +1,4 @@
11
---
2-
title: Using user-defined schemas
3-
description: Tips for using T-SQL user-defined schemas in Azure SQL Data Warehouse for developing solutions.
4-
services: sql-data-warehouse
5-
author: XiaoyuMSFT
6-
manager: craigg
7-
ms.service: sql-data-warehouse
8-
ms.topic: conceptual
9-
ms.subservice: development
10-
ms.date: 04/17/2018
11-
ms.author: xiaoyul
12-
ms.reviewer: igorstan
13-
ms.custom: seo-lt-2019
2+
redirect_url: /azure/synapse-analytics/sql-data-warehouse/sql-data-warehouse-develop-user-defined-schemas
3+
redirect_document_id: true
144
---
15-
16-
# Using user-defined schemas in SQL Data Warehouse
17-
Tips for using T-SQL user-defined schemas in Azure SQL Data Warehouse for developing solutions.
18-
19-
## Schemas for application boundaries
20-
21-
Traditional data warehouses often use separate databases to create application boundaries based on either workload, domain or security. For example, a traditional SQL Server data warehouse might include a staging database, a data warehouse database, and some data mart databases. In this topology each database operates as a workload and security boundary in the architecture.
22-
23-
By contrast, SQL Data Warehouse runs the entire data warehouse workload within one database. Cross database joins are not permitted. Therefore SQL Data Warehouse expects all tables used by the warehouse to be stored within the one database.
24-
25-
> [!NOTE]
26-
> SQL Data Warehouse does not support cross database queries of any kind. Consequently, data warehouse implementations that leverage this pattern will need to be revised.
27-
>
28-
>
29-
30-
## Recommendations
31-
These are recommendations for consolidating workloads, security, domain and functional boundaries by using user defined schemas
32-
33-
1. Use one SQL Data Warehouse database to run your entire data warehouse workload
34-
2. Consolidate your existing data warehouse environment to use one SQL Data Warehouse database
35-
3. Leverage **user-defined schemas** to provide the boundary previously implemented using databases.
36-
37-
If user-defined schemas have not been used previously then you have a clean slate. Simply use the old database name as the basis for your user-defined schemas in the SQL Data Warehouse database.
38-
39-
If schemas have already been used then you have a few options:
40-
41-
1. Remove the legacy schema names and start fresh
42-
2. Retain the legacy schema names by pre-pending the legacy schema name to the table name
43-
3. Retain the legacy schema names by implementing views over the table in an extra schema to re-create the old schema structure.
44-
45-
> [!NOTE]
46-
> On first inspection option 3 may seem like the most appealing option. However, the devil is in the detail. Views are read only in SQL Data Warehouse. Any data or table modification would need to be performed against the base table. Option 3 also introduces a layer of views into your system. You might want to give this some additional thought if you are using views in your architecture already.
47-
>
48-
>
49-
50-
### Examples:
51-
Implement user-defined schemas based on database names
52-
53-
```sql
54-
CREATE SCHEMA [stg]; -- stg previously database name for staging database
55-
GO
56-
CREATE SCHEMA [edw]; -- edw previously database name for the data warehouse
57-
GO
58-
CREATE TABLE [stg].[customer] -- create staging tables in the stg schema
59-
( CustKey BIGINT NOT NULL
60-
, ...
61-
);
62-
GO
63-
CREATE TABLE [edw].[customer] -- create data warehouse tables in the edw schema
64-
( CustKey BIGINT NOT NULL
65-
, ...
66-
);
67-
```
68-
69-
Retain legacy schema names by pre-pending them to the table name. Use schemas for the workload boundary.
70-
71-
```sql
72-
CREATE SCHEMA [stg]; -- stg defines the staging boundary
73-
GO
74-
CREATE SCHEMA [edw]; -- edw defines the data warehouse boundary
75-
GO
76-
CREATE TABLE [stg].[dim_customer] --pre-pend the old schema name to the table and create in the staging boundary
77-
( CustKey BIGINT NOT NULL
78-
, ...
79-
);
80-
GO
81-
CREATE TABLE [edw].[dim_customer] --pre-pend the old schema name to the table and create in the data warehouse boundary
82-
( CustKey BIGINT NOT NULL
83-
, ...
84-
);
85-
```
86-
87-
Retain legacy schema names using views
88-
89-
```sql
90-
CREATE SCHEMA [stg]; -- stg defines the staging boundary
91-
GO
92-
CREATE SCHEMA [edw]; -- stg defines the data warehouse boundary
93-
GO
94-
CREATE SCHEMA [dim]; -- edw defines the legacy schema name boundary
95-
GO
96-
CREATE TABLE [stg].[customer] -- create the base staging tables in the staging boundary
97-
( CustKey BIGINT NOT NULL
98-
, ...
99-
)
100-
GO
101-
CREATE TABLE [edw].[customer] -- create the base data warehouse tables in the data warehouse boundary
102-
( CustKey BIGINT NOT NULL
103-
, ...
104-
)
105-
GO
106-
CREATE VIEW [dim].[customer] -- create a view in the legacy schema name boundary for presentation consistency purposes only
107-
AS
108-
SELECT CustKey
109-
, ...
110-
FROM [edw].customer
111-
;
112-
```
113-
114-
> [!NOTE]
115-
> Any change in schema strategy needs a review of the security model for the database. In many cases you might be able to simplify the security model by assigning permissions at the schema level. If more granular permissions are required then you can use database roles.
116-
>
117-
>
118-
119-
## Next steps
120-
For more development tips, see [development overview](sql-data-warehouse-overview-develop.md).
121-

0 commit comments

Comments
 (0)