Skip to content

Commit 8424db4

Browse files
dimitri-furmandimitri-furman
committed
Additional columnstore articles
1 parent 8166d2b commit 8424db4

File tree

2 files changed

+278
-275
lines changed

2 files changed

+278
-275
lines changed

‎docs/relational-databases/indexes/columnstore-indexes-data-warehouse.md

Lines changed: 96 additions & 96 deletions
Original file line numberDiff line numberDiff line change
@@ -3,128 +3,128 @@ title: "Columnstore indexes in data warehousing"
33
description: Learn more about how to benefit from columnstore indexes in data warehousing with the SQL Database Engine.
44
author: MikeRayMSFT
55
ms.author: mikeray
6-
ms.date: 02/28/2025
6+
ms.date: 04/04/2025
77
ms.service: sql
88
ms.subservice: table-view-index
99
ms.topic: conceptual
1010
ms.custom:
1111
- ignite-2024
1212
monikerRange: ">=aps-pdw-2016 || =azuresqldb-current || >=sql-server-2016 || >=sql-server-linux-2017 || =azuresqldb-mi-current || =fabric"
1313
---
14+
1415
# Columnstore indexes in data warehousing
16+
1517
[!INCLUDE [SQL Server Azure SQL Database PDW FabricSQLDB](../../includes/applies-to-version/sql-asdb-asdbmi-pdw-fabricsqldb.md)]
1618

17-
Columnstore indexes, in conjunction with partitioning, are essential for building a [!INCLUDE [ssNoVersion](../../includes/ssnoversion-md.md)] data warehouse. This article focuses on key use cases and examples for data warehousing designs with the SQL Database Engine.
18-
19+
Columnstore indexes, in conjunction with partitioning, are essential for building a [!INCLUDE [ssNoVersion](../../includes/ssnoversion-md.md)] data warehouse. This article focuses on key use cases and examples for data warehousing designs with the SQL Database Engine.
20+
1921
## Key features for data warehousing
2022

21-
[!INCLUDE [sssql16-md](../../includes/sssql16-md.md)] introduced these features for columnstore performance enhancements:
22-
23-
- Always On supports querying a columnstore index on a readable secondary replica.
24-
- Multiple Active Result Sets (MARS) supports columnstore indexes.
25-
- A new dynamic management view [sys.dm_db_column_store_row_group_physical_stats (Transact-SQL)](../system-dynamic-management-views/sys-dm-db-column-store-row-group-physical-stats-transact-sql.md) provides performance troubleshooting information at the row group level.
26-
- Single-threaded queries on columnstore indexes can run in batch mode. Previously, only multi-threaded queries could run in batch mode.
27-
- The `SORT` operator runs in batch mode.
28-
- Multiple `DISTINCT` operations run in batch mode.
29-
- Window Aggregates now runs in batch mode for database compatibility level 130 and higher.
30-
- Aggregate Pushdown for efficient processing of aggregates. This is supported on all database compatibility levels.
31-
- String predicate pushdown for efficient processing of string predicates. This is supported on all database compatibility levels.
32-
- Snapshot isolation for database compatibility level 130 and higher.
33-
- Ordered clustered columnstore indexes were introduced with [!INCLUDE [sql-server-2022](../../includes/sssql22-md.md)]. For more information, see [CREATE COLUMNSTORE INDEX](../../t-sql/statements/create-columnstore-index-transact-sql.md#order-for-clustered-columnstore) and [Performance tuning with ordered columnstore indexes](ordered-columnstore-indexes.md). For ordered columnstore index availability, see [Ordered column index availability](columnstore-indexes-overview.md#ordered-columnstore-index-availability).
23+
[!INCLUDE [sssql16-md](../../includes/sssql16-md.md)] introduced these features for columnstore performance enhancements:
24+
25+
- Always On supports querying a columnstore index on a readable secondary replica.
26+
- Multiple Active Result Sets (MARS) supports columnstore indexes.
27+
- A new dynamic management view [sys.dm_db_column_store_row_group_physical_stats (Transact-SQL)](../system-dynamic-management-views/sys-dm-db-column-store-row-group-physical-stats-transact-sql.md) provides performance troubleshooting information at the row group level.
28+
- Serial queries on columnstore indexes can run in batch mode. Previously, only parallel queries could run in batch mode.
29+
- The *sort* operator runs in batch mode.
30+
- Multiple *distinct* operations run in batch mode.
31+
- Window aggregates now runs in batch mode for database compatibility level 130 and higher.
32+
- Aggregate pushdown for efficient processing of aggregates. This is supported on all database compatibility levels.
33+
- String predicate pushdown for efficient processing of string predicates. This is supported on all database compatibility levels.
34+
- Snapshot isolation for database compatibility level 130 and higher.
35+
- Ordered clustered columnstore indexes were introduced with [!INCLUDE [sql-server-2022](../../includes/sssql22-md.md)]. For more information, see [CREATE COLUMNSTORE INDEX](../../t-sql/statements/create-columnstore-index-transact-sql.md#order-for-clustered-columnstore) and [Performance tuning with ordered columnstore indexes](ordered-columnstore-indexes.md). For ordered columnstore index availability, see [Ordered column index availability](columnstore-indexes-overview.md#ordered-columnstore-index-availability).
3436

3537
For more information about new features in versions and platforms of SQL Server and Azure SQL, see [What's new in columnstore indexes](columnstore-indexes-what-s-new.md).
36-
38+
3739
## Improve performance by combining nonclustered and columnstore indexes
38-
Starting with [!INCLUDE [sssql16-md](../../includes/sssql16-md.md)], you can define rowstore nonclustered indexes on a clustered columnstore index.
39-
40+
41+
Starting with [!INCLUDE [sssql16-md](../../includes/sssql16-md.md)], you can create rowstore nonclustered indexes on a clustered columnstore index.
42+
4043
### Example: Improve efficiency of table seeks with a nonclustered index
41-
To improve efficiency of table seeks in a data warehouse, you can create a nonclustered index designed to run queries that perform best with table seeks. For example, queries that look for matching values or return a small range of values perform better against a B-tree index rather than a columnstore index. They don't require a full table scan through the columnstore index and return the correct result faster by doing a binary search through a B-tree index.
42-
43-
```sql
44-
--BASIC EXAMPLE: Create a nonclustered index on a columnstore table.
45-
46-
--Create the table
47-
CREATE TABLE t_account (
48-
AccountKey int NOT NULL,
49-
AccountDescription nvarchar (50),
50-
AccountType nvarchar(50),
51-
UnitSold int
52-
);
53-
GO
54-
44+
45+
To improve efficiency of table seeks in a data warehouse, you can create a nonclustered index designed to run queries that perform best with table seeks. For example, queries that look for matching values or return a small range of values perform better against a B-tree index rather than a columnstore index. They don't require a full scan of the columnstore index and return the correct result faster by doing a binary search through a B-tree index.
46+
47+
```sql
48+
--BASIC EXAMPLE: Create a nonclustered index on a columnstore table.
49+
50+
--Create the table
51+
CREATE TABLE t_account (
52+
AccountKey int NOT NULL,
53+
AccountDescription nvarchar (50),
54+
AccountType nvarchar(50),
55+
UnitSold int
56+
);
57+
5558
--Store the table as a columnstore.
56-
CREATE CLUSTERED COLUMNSTORE INDEX taccount_cci ON t_account;
57-
GO
58-
59-
--Add a nonclustered index.
60-
CREATE UNIQUE INDEX taccount_nc1 ON t_account (AccountKey);
61-
```
62-
59+
CREATE CLUSTERED COLUMNSTORE INDEX taccount_cci ON t_account;
60+
61+
--Add a nonclustered index.
62+
CREATE UNIQUE INDEX taccount_nc1 ON t_account (AccountKey);
63+
```
64+
6365
### Example: Use a nonclustered index to enforce a primary key constraint on a columnstore table
6466

65-
By design, a columnstore table doesn't allow a clustered primary key constraint. Now you can use a nonclustered index on a columnstore table to enforce a primary key constraint. A primary key is equivalent to a UNIQUE constraint on a non-NULL column, and [!INCLUDE [ssNoVersion](../../includes/ssnoversion-md.md)] implements a UNIQUE constraint as a nonclustered index. Combining these facts, the following example defines a UNIQUE constraint on the non-NULL column accountkey. The result is a nonclustered index that enforces a primary key constraint as a UNIQUE constraint on a non-NULL column.
66-
67-
Next, the table is converted to a clustered columnstore index. During the conversion, the nonclustered index persists. The result is a clustered columnstore index with a nonclustered index that enforces a primary key constraint. Since any update or insert on the columnstore table also affects the nonclustered index, all operations that violate the unique constraint and the non-NULL causes the entire operation to fail.
68-
69-
The result is a columnstore index with a nonclustered index that enforces a primary key constraint on both indexes.
70-
67+
By design, a columnstore table doesn't allow a clustered primary key constraint. Now you can use a nonclustered index on a columnstore table to enforce uniqueness. A primary key is equivalent to a `UNIQUE` constraint on a non-NULL column, and [!INCLUDE [ssNoVersion](../../includes/ssnoversion-md.md)] implements a `UNIQUE` constraint as a nonclustered index. Combining these facts, the following example defines a `UNIQUE` constraint on the non-NULL column `AccountKey`. The result is a nonclustered index that enforces uniquness on a non-NULL column.
68+
69+
Next, the table is converted to a clustered columnstore index. During the conversion, the nonclustered index persists. The result is a clustered columnstore index with a nonclustered index that enforces uniqueness. Since any update or insert on the columnstore table also affects the nonclustered index, all operations that violate the unique constraint and the non-NULL constraint cause the entire operation to fail.
70+
71+
The result is a columnstore index with a nonclustered index that enforces uniqueness on both indexes.
72+
7173
```sql
72-
--EXAMPLE: Enforce a primary key constraint on a columnstore table.
73-
74-
--Create a rowstore table with a unique constraint.
75-
--The unique constraint is implemented as a nonclustered index.
76-
CREATE TABLE t_account (
77-
AccountKey int NOT NULL,
78-
AccountDescription nvarchar (50),
79-
AccountType nvarchar(50),
80-
UnitSold int,
81-
82-
CONSTRAINT uniq_account UNIQUE (AccountKey)
83-
);
84-
85-
--Store the table as a columnstore.
86-
--The unique constraint is preserved as a nonclustered index on the columnstore table.
87-
CREATE CLUSTERED COLUMNSTORE INDEX t_account_cci ON t_account
88-
89-
--By using the previous two steps, every row in the table meets the UNIQUE constraint
90-
--on a non-NULL column.
91-
--This has the same end-result as having a primary key constraint
92-
--All updates and inserts must meet the unique constraint on the nonclustered index or they will fail.
74+
--EXAMPLE: Enforce a primary key constraint on a columnstore table.
75+
76+
--Create a rowstore table with a unique constraint.
77+
--The unique constraint is implemented as a nonclustered index.
78+
CREATE TABLE t_account (
79+
AccountKey int NOT NULL,
80+
AccountDescription nvarchar (50),
81+
AccountType nvarchar(50),
82+
UnitSold int,
83+
CONSTRAINT uniq_account UNIQUE (AccountKey)
84+
);
85+
86+
--Store the table as a columnstore.
87+
--The unique constraint is preserved as a nonclustered index on the columnstore table.
88+
CREATE CLUSTERED COLUMNSTORE INDEX t_account_cci ON t_account;
89+
90+
--By using the previous two steps, every row in the table meets the UNIQUE constraint
91+
--on a non-NULL column.
92+
--This has the same end-result as having a primary key constraint
93+
--All updates and inserts must meet the unique constraint on the nonclustered index or they will fail.
9394

94-
--If desired, add a foreign key constraint on AccountKey.
95+
--If desired, add a foreign key constraint on AccountKey.
9596

9697
ALTER TABLE [dbo].[t_account]
97-
WITH CHECK ADD FOREIGN KEY([AccountKey]) REFERENCES my_dimension(Accountkey);
98-
```
99-
100-
### Improve performance by enabling row-level and row-group-level locking
98+
WITH CHECK ADD FOREIGN KEY([AccountKey]) REFERENCES my_dimension (Accountkey);
99+
```
100+
101+
### Improve performance by enabling row-level and rowgroup-level locking
102+
103+
To complement the nonclustered index on a columnstore index feature, [!INCLUDE [sssql16-md](../../includes/sssql16-md.md)] offers granular locking capability for `SELECT`, `UPDATE`, and `DELETE` operations. Queries can run with row-level locking on index seeks against a nonclustered index and rowgroup-level locking on full table scans against the columnstore index. Use this to achieve higher read/write concurrency by using row-level and rowgroup-level locking appropriately.
104+
105+
```sql
106+
--Granular locking example
107+
--Store table t_account as a columnstore table.
108+
CREATE CLUSTERED COLUMNSTORE INDEX taccount_cci ON t_account
109+
110+
--Add a nonclustered index for use with this example
111+
CREATE UNIQUE INDEX taccount_nc1 ON t_account (AccountKey);
112+
113+
--Look at locking with access through the nonclustered index
114+
SET TRANSACTION ISOLATION LEVEL REPEATABLE READ;
115+
116+
BEGIN TRAN
117+
-- The query plan chooses a seek operation on the nonclustered index
118+
-- and takes the row lock
119+
SELECT *
120+
FROM t_account
121+
WHERE AccountKey = 100;
122+
COMMIT TRAN;
123+
```
101124

102-
To complement the nonclustered index on a columnstore index feature, [!INCLUDE [sssql16-md](../../includes/sssql16-md.md)] offers granular locking capability for select, update, and delete operations. Queries can run with row-level locking on index seeks against a nonclustered index and rowgroup-level locking on full table scans against the columnstore index. Use this to achieve higher read/write concurrency by using row-level and rowgroup-level locking appropriately.
103-
104-
```sql
105-
--Granular locking example
106-
--Store table t_account as a columnstore table.
107-
CREATE CLUSTERED COLUMNSTORE INDEX taccount_cci ON t_account
108-
GO
109-
110-
--Add a nonclustered index for use with this example
111-
CREATE UNIQUE INDEX taccount_nc1 ON t_account (AccountKey);
112-
GO
113-
114-
--Look at locking with access through the nonclustered index
115-
SET TRANSACTION ISOLATION LEVEL repeatable read;
116-
GO
117-
118-
BEGIN TRAN
119-
-- The query plan chooses a seek operation on the nonclustered index
120-
-- and takes the row lock
121-
SELECT * FROM t_account WHERE AccountKey = 100;
122-
COMMIT TRAN
123-
```
124-
125125
### Snapshot isolation and read-committed snapshot isolation
126126

127-
Use snapshot isolation (SI) to guarantee transactional consistency, and read-committed snapshot isolation (RCSI) to guarantee statement level consistency for queries on columnstore indexes. This allows the queries to run without blocking data writers. This non-blocking behavior also significantly reduces the likelihood of deadlocks for complex transactions. For more information, see [Snapshot Isolation in SQL Server](../../t-sql/statements/set-transaction-isolation-level-transact-sql.md#arguments).
127+
Use snapshot isolation (SI) to guarantee transactional consistency, and read-committed snapshot isolation (RCSI) to guarantee statement level consistency for queries on columnstore indexes. This allows the queries to run without blocking data writers. This non-blocking behavior also significantly reduces the likelihood of deadlocks for complex transactions. For more information, see [Row versioning-based isolation levels in the Database Engine](../sql-server-transaction-locking-and-row-versioning-guide.md#Row_versioning).
128128

129129
## Related content
130130

0 commit comments

Comments
 (0)