Skip to content

Commit b87f190

Browse files
AbhinavTripsAbhinavTrips
committed
mv changes added back
1 parent c584790 commit b87f190

File tree

1 file changed

+309
-0
lines changed

1 file changed

+309
-0
lines changed

‎articles/cosmos-db/nosql/materialized-views.md

Lines changed: 309 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,309 @@
1+
---
2+
title: Materialized views (preview)
3+
titleSuffix: Azure Cosmos DB for NoSQL
4+
description: Learn how to efficiently query a base container by using predefined filters in materialized views for Azure Cosmos DB for NoSQL. Use materilaized views as global secondary indexes to avoid expensive cross-partition queries.
5+
author: AbhinavTrips
6+
ms.author: abtripathi
7+
ms.reviewer: sidandrews
8+
ms.service: cosmos-db
9+
ms.subservice: nosql
10+
ms.custom: build-2023, devx-track-azurecli
11+
ms.topic: how-to
12+
ms.date: 06/09/2023
13+
---
14+
15+
# Materialized views for Azure Cosmos DB for NoSQL (preview)
16+
17+
[!INCLUDE[NoSQL](../includes/appliesto-nosql.md)]
18+
19+
> [!IMPORTANT]
20+
> The materialized view feature of Azure Cosmos DB for NoSQL is currently in preview. You can enable this feature by using the Azure portal. This preview is provided without a service-level agreement. At this time, we don't recommend that you use materialized views for production workloads. Certain features of this preview might not be supported or might have constrained capabilities. For more information, see the [supplemental terms of use for Microsoft Azure previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).
21+
22+
Applications frequently are required to make queries that don't specify a partition key. In these cases, the queries might scan through all data for a small result set. The queries end up being expensive because they inadvertently run as a cross-partition query.
23+
24+
Materialized views, when defined, help provide a way to efficiently query a base container in Azure Cosmos DB by using filters that don't include the partition key. When users write to the base container, the materialized view is built automatically in the background. This view can have a different partition key for efficient lookups. The view also contains only fields that are explicitly projected from the base container. This view is a read-only table. The Azure Cosmos DB materialized views can be used as global secondary indexes to avoid expensive cross-partition queries.
25+
26+
> [!IMPORTANT]
27+
> The materialized view feature of Azure Cosmos DB for NoSQL can be used as Global Secondary Indexes. Users can specify the fields that are projected from the base container to the materialized view and they can choose a different partition key for the materialized view. Choosing a different partition key based on the most common queries, helps in scoping the queries to a single logical partition and avoiding cross-partition queries..
28+
29+
With a materialized view, you can:
30+
31+
- Use the view as a lookup or mapping container to persist cross-partition scans that would otherwise be expensive queries.
32+
- Provide a SQL-based predicate (without conditions) to populate only specific fields.
33+
- Use change feed triggers to create real-time views to simplify event-based scenarios that are commonly stored as separate containers.
34+
35+
The benefits of using Azure Cosmos DB Materiliazed Views include, but aren't limited to:
36+
37+
- You can implement server-side denormalization by using materialized views. With server-side denormalization, you can avoid multiple independent tables and computationally complex denormalization in client applications.
38+
- Materialized views automatically update views to keep views consistent with the base container. This automatic update abstracts the responsibilities of your client applications that would otherwise typically implement custom logic to perform dual writes to the base container and the view.
39+
- Materialized views optimize read performance by reading from a single view.
40+
- You can specify throughput for the materialized view independently.
41+
- You can configure a materialized view builder layer to map to your requirements to hydrate a view.
42+
- Materialized views improve write performance (compared to a multi-container-write strategy) because write operations need to be written only to the base container.
43+
- The Azure Cosmos DB implementation of materialized views is based on a pull model. This implementation doesn't affect write performance.
44+
- Azure Cosmos DB materialized views for NoSQL API caters to the Global Secondary Index use cases as well. Global Secondary Indexes are also used to maintain secondary data views and help in reducing cross-partition queries.
45+
46+
## Prerequisites
47+
48+
- An existing Azure Cosmos DB account.
49+
- If you have an Azure subscription, [create a new account](how-to-create-account.md?tabs=azure-portal).
50+
- If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.
51+
- Alternatively, you can [try Azure Cosmos DB free](../try-free.md) before you commit.
52+
53+
## Enable materialized views
54+
55+
Use the Azure CLI to enable the materialized views feature either by using a native command or a REST API operation on your Cosmos DB for NoSQL account.
56+
57+
### [Azure portal](#tab/azure-portal)
58+
59+
1. Sign in to the [Azure portal](https://portal.azure.com/).
60+
61+
1. Go to your API for NOSQL account.
62+
63+
1. In the resource menu, select **Settings**.
64+
65+
1. In the **Features** section under **Settings**, toggle the **Materialized View for NoSQL API (Preview)** option to **On**.
66+
67+
1. In the new dialog, select **Enable** to enable this feature for the account.
68+
69+
### [Azure CLI](#tab/azure-cli)
70+
71+
1. Sign in to the Azure CLI.
72+
73+
```azurecli
74+
az login
75+
```
76+
77+
> [!NOTE]
78+
> If you need to first install the Azure CLI, see [How to install the Azure CLI](/cli/azure/install-azure-cli).
79+
80+
1. Define the variables for the resource group and account name for your existing API for NoSQL account.
81+
82+
```azurecli
83+
# Variable for resource group name
84+
resourceGroupName="<resource-group-name>"
85+
86+
# Variable for account name
87+
accountName="<account-name>"
88+
89+
# Variable for Azure subscription
90+
subscriptionId="<subscription-id>"
91+
```
92+
93+
1. Create a new JSON file named *capabilities.json* by using the capabilities manifest.
94+
95+
```json
96+
{
97+
"properties": {
98+
"enableMaterializedViews": true
99+
}
100+
}
101+
```
102+
103+
1. Get the identifier of the account and store it in a shell variable named `$accountId`.
104+
105+
```azurecli
106+
accountId="/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.DocumentDB/databaseAccounts/$accountName"
107+
```
108+
109+
1. Enable the preview materialized views feature for the account by using the REST API and [az rest](/cli/azure/reference-index#az-rest) with an HTTP `PATCH` verb.
110+
111+
```azurecli
112+
az rest \
113+
--method PATCH \
114+
--uri "https://management.azure.com/$accountId?api-version=2022-11-15-preview" \
115+
--body @capabilities.json
116+
```
117+
118+
---
119+
120+
## Create a materialized view builder
121+
122+
Create a materialized view builder to automatically transform data and write to a materialized view.
123+
124+
### [Azure portal](#tab/azure-portal)
125+
126+
1. Sign in to the [Azure portal](https://portal.azure.com/).
127+
128+
1. Go to your API for NoSQL account.
129+
130+
1. In the resource menu, select **Materialized Views Builder**.
131+
132+
1. On the **Materialized Views Builder** page, configure the SKU and the number of instances for the builder.
133+
134+
> [!NOTE]
135+
> This resource menu option and page appear only when the materialized views feature is enabled for the account.
136+
137+
1. Select **Save**.
138+
139+
### [Azure CLI](#tab/azure-cli)
140+
141+
1. Create a new JSON file named *builder.json* by using the builder manifest:
142+
143+
```json
144+
{
145+
"properties": {
146+
"serviceType": "materializedViewsBuilder",
147+
"instanceCount": 1,
148+
"instanceSize": "Cosmos.D4s"
149+
}
150+
}
151+
```
152+
153+
1. Enable the materialized views builder for the account by using the REST API and `az rest` with an HTTP `PUT` verb:
154+
155+
```azurecli
156+
az rest \
157+
--method PUT \
158+
--uri "https://management.azure.com$accountId/services/materializedViewsBuilder?api-version=2022-11-15-preview" \
159+
--body @builder.json
160+
```
161+
162+
1. Wait for a couple of minutes, and then check the status by using `az rest` again with the HTTP `GET` verb. The status in the output should now be `Running`.
163+
164+
```azurecli
165+
az rest \
166+
--method GET \
167+
--uri "https://management.azure.com$accountId/services/materializedViewsBuilder?api-version=2022-11-15-preview"
168+
```
169+
170+
---
171+
172+
Azure Cosmos DB For NoSQL uses a materialized view builder compute layer to maintain the views.
173+
174+
You have the flexibility of configuring the view builder's compute instances based on your latency and lag requirements to hydrate the views. From a technical standpoint, this compute layer helps you manage connections between partitions in a more efficient manner, even when the data size is large and the number of partitions is high.
175+
176+
The compute containers are shared among all materialized views within an Azure Cosmos DB account. Each provisioned compute container initiates multiple tasks that read the change feed from the base container partitions and write data to the target materialized view or views. The compute container transforms the data per the materialized view definition for each materialized view in the account.
177+
178+
## Create a materialized view
179+
180+
After your account and the materialized view builder are set up, you should be able to create materialized views by using the REST API.
181+
182+
### [Azure portal / Azure CLI](#tab/azure-portal+azure-cli)
183+
184+
1. Use the Azure portal, the Azure SDK, the Azure CLI, or the REST API to create a source container that has `/accountId` as the partition key path. Name this source container `mv-src.
185+
186+
> [!NOTE]
187+
> The `/accountId` field is used only as an example in this article. For your own containers, select a partition key that works for your solution.
188+
189+
1. Insert a few items in the source container. To follow the examples that are shown in this article, make sure that the items have `accountId`, `fullName`, and `emailAddress` fields. A sample item might look like this example:
190+
191+
```json
192+
{
193+
"accountId": "prikrylova-libuse",
194+
"emailAddress": "[email protected]",
195+
"name": {
196+
"first": "Libuse",
197+
"last": "Prikrylova"
198+
}
199+
}
200+
```
201+
202+
> [!NOTE]
203+
> In this example, you populate the source container with sample data. You can also create a materialized view from an empty source container.
204+
205+
1. Now, create a materialized view named `mv-target` with a partition key path that is different from the source container. For this example, specify `/emailAddress` as the partition key path for the `mv-target` container.
206+
207+
1. First, create a definition manifest for a materialized view and save it in a JSON file named *definition.json*:
208+
209+
```json
210+
{
211+
"location": "North Central US",
212+
"tags": {},
213+
"properties": {
214+
"resource": {
215+
"id": "mv-target",
216+
"partitionKey": {
217+
"paths": [
218+
"/emailAddress"
219+
],
220+
"kind": "Hash"
221+
},
222+
"materializedViewDefinition": {
223+
"sourceCollectionId": "mv-src",
224+
"definition": "SELECT s.accountId, s.emailAddress FROM s"
225+
}
226+
},
227+
"options": {
228+
"throughput": 400
229+
}
230+
}
231+
}
232+
```
233+
234+
> [!NOTE]
235+
> In the template, notice that the partitionKey path is set as `/emailAddress`. We also have more parameters to specify the source collection and the definition to populate the materialized view.
236+
237+
1. Now, make a REST API call to create the materialized view as defined in the *mv_definition.json* file. Use the Azure CLI to make the REST API call.
238+
239+
1. Create a variable for the name of the materialized view and source database name:
240+
241+
```azurecli
242+
materializedViewName="mv-target"
243+
244+
# Variable for database name used in later section
245+
databaseName="<database-that-contains-source-collection>"
246+
```
247+
248+
1. If you haven't already, get the identifier of the account and store it in a shell variable named `$accountId`.
249+
250+
```azurecli
251+
accountId="/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.DocumentDB/databaseAccounts/$accountName"
252+
```
253+
254+
1. Make a REST API call to create the materialized view:
255+
256+
```azurecli
257+
az rest \
258+
--method PUT \
259+
--uri "https://management.azure.com$accountId/sqlDatabases/";\
260+
"$databaseName/containers/$materializedViewName?api-version=2022-11-15-preview" \
261+
--body @definition.json \
262+
--headers content-type=application/json
263+
```
264+
265+
1. Check the status of the materialized view container creation by using the REST API:
266+
267+
```azurecli
268+
az rest \
269+
--method GET \
270+
--uri "https://management.azure.com$accountId/sqlDatabases/";\
271+
"$databaseName/containers/$materializedViewName?api-version=2022-11-15-preview" \
272+
--headers content-type=application/json \
273+
--query "{mvCreateStatus: properties.Status}"
274+
```
275+
276+
---
277+
278+
After the materialized view is created, the materialized view container automatically syncs changes with the source container. Try executing create, read, update, and delete (CRUD) operations in the source container. You'll see the same changes in the materialized view container.
279+
280+
> [!NOTE]
281+
> Materialized view containers are read-only containers for users. The containers can be automatically modified only by a materialized view builder.
282+
283+
## Current limitations
284+
285+
There are a few limitations with the Azure Cosmos DB for NoSQL API materialized view feature while it is in preview:
286+
287+
- `WHERE` clauses aren't supported in the materialized view definition.
288+
- You can project only the source container item's JSON `object` property list in the materialized view definition. Currently, the list can contain only one level of properties in the JSON tree.
289+
- In the materialized view definition, aliases aren't supported for fields of documents.
290+
- We recommend that you create a materialized view when the source container is still empty or has only a few items.
291+
- Restoring a container from a backup doesn't restore materialized views. You must re-create the materialized views after the restore process is finished.
292+
- You must delete all materialized views that are defined on a specific source container before you delete the source container.
293+
- Point-in-time restore, hierarchical partitioning, and end-to-end encryption aren't supported on source containers that have materialized views associated with them.
294+
- Role-based access control is currently not supported for materialized views.
295+
- Cross-tenant customer-managed key (CMK) encryption isn't supported on materialized views.
296+
- Currently, this feature can't be enabled if any of the following features are enabled: partition merge, analytical store, or continuous backup.
297+
298+
Note the additional following limitations:
299+
300+
- Availability zones
301+
- Materialized views can't be enabled on an account that has availability zone-enabled regions.
302+
- Adding a new region with an availability zone isn't supported after `enableMaterializedViews` is set to `true` on the account.
303+
- Periodic backup and restore
304+
- Materialized views aren't automatically restored by using the restore process. You must re-create the materialized views after the restore process is finished. Then, you should configure `enableMaterializedViews` on the restored account before you create the materialized views and builders again.
305+
306+
## Next steps
307+
308+
> [!div class="nextstepaction"]
309+
> [Data modeling and partitioning](model-partition-example.md)

0 commit comments

Comments
 (0)