You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
> Materialized Views for Azure Cosmos DB API for Cassandra is currently in preview. You can enable this feature from the portal.
17
-
18
+
> Materialized Views for API for Cassandra is currently in preview. You can enable this feature from the portal.
19
+
>
18
20
> Materialized View preview version is provided without a service level agreement, and it's not recommended for production workloads. Certain features might not be supported or might have constrained capabilities.
21
+
>
19
22
> For more information, see [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).
20
23
21
24
## Feature overview
@@ -24,33 +27,33 @@ Materialized Views when defined will help provide a means to efficiently query a
24
27
25
28
You can query a column store without specifying a partition key by using Secondary Indexes. However, the query won't be effective for columns with high cardinality (scanning through all data for a small result set) or columns with low cardinality. Such queries end up being expensive as they end up being a cross partition query.
26
29
27
-
With Materialized view, you can
28
-
- Use as Lookup table or Mapping table and save cross partition scans that reduce expensive queries
29
-
- Provide SQL based conditional predicate to populate only certain columns and certain data that meet the pre-condition
30
-
- Real time MVs that simplify real time event based scenarios where customers today use Change feed trigger for precondition checks to populate new collections"
30
+
With Materialized view, you can:
31
+
32
+
- Use as Lookup table or Mapping table and save cross partition scans that reduce expensive queries.
33
+
- Provide SQL based conditional predicate to populate only certain columns and certain data that meet the pre-condition.
34
+
- Real time MVs that simplify real time event based scenarios where customers today use Change feed trigger for precondition checks to populate new collections".
31
35
32
36
## Main benefits
33
37
34
-
- With Materialized View (Server side denormalization), you can avoid multiple independent tables and client side denormalization.
38
+
- With Materialized View (Server side denormalization), you can avoid multiple independent tables and client side denormalization.
35
39
- Materialized view feature takes on the responsibility of updating views in order to keep them consistent with the base table. With this feature, you can avoid dual writes to the base table and the view.
- Ability to specify throughput for the materialized view independently
38
42
- Based on the requirements to hydrate the view, you can configure the MV builder layer appropriately.
39
43
- Speeding up write operations as it only needs to be written to the base table.
40
44
- Additionally, This implementation on Cosmos DB is based on a pull model, which doesn't affect the writer performance.
41
45
42
-
43
-
44
46
## How to get started?
45
47
46
-
New Cassandra API accounts with Materialized Views enabled can be provisioned on your subscription by using REST API calls from az CLI.
48
+
New Cassandra API accounts with Materialized Views enabled can be provisioned on your subscription by using REST API calls from Azure CLI.
47
49
48
50
### Log in to the Azure command line interface
49
51
50
-
Install Azure CLI as mentioned at [How to install the Azure CLI | Microsoft Docs](/cli/azure/install-azure-cli) and log on using the below:
51
-
```azurecli-interactive
52
-
az login
53
-
```
52
+
Install Azure CLI as mentioned at [How to install the Azure CLI | Microsoft Docs](/cli/azure/install-azure-cli) and log in:
53
+
54
+
```azurecli
55
+
az login
56
+
```
54
57
55
58
### Enable Materialized view feature on DB account
56
59
@@ -68,100 +71,97 @@ Install Azure CLI as mentioned at [How to install the Azure CLI | Microsoft Docs
68
71
69
72
:::image type="content" source="./media/materialized-view/Enable_mv_on_portal.png" alt-text="Screenshot to enable Materialized view feature on portal Azure Cosmos DB API for Cassandra":::
70
73
71
-
72
-
#### Using AZ CLI
74
+
#### Using Azure CLI
73
75
74
76
Please use this link to install Microsoft Azure CLI 'cosmosdb-preview' Extension: [Azure CLI extension overview](https://learn.microsoft.com/cli/azure/azure-cli-extensions-overview)
75
77
76
-
Refer to this link on documentation for AZ CLI command for enabling Materialized View on account: [AZ CLI documentation](https://github.com/Azure/azure-cli-extensions/tree/main/src/cosmosdb-preview#enable-materialized-views-on-a-existing-cosmosdb-account)
78
+
Refer to this link on documentation for Azure CLI command for enabling Materialized View on account: [Azure CLI documentation](https://github.com/Azure/azure-cli-extensions/tree/main/src/cosmosdb-preview#enable-materialized-views-on-a-existing-cosmosdb-account)
77
79
78
-
Enable Materialized View for account using command as in below example
Azure Cosmos DB Cassandra API uses a MV builder compute layer to maintain Materialized views.
107
+
Azure Cosmos DB Cassandra API uses a MV builder compute layer to maintain Materialized views.
111
108
112
-
Customer gets flexibility to configure the MV builder compute instances depending on the latency and lag requirements to hydrate the views. From a technical stand point, this compute layer helps manage connections between partitions in a more efficient manner even when the data size is huge and the number of partitions is high.
109
+
Customer gets flexibility to configure the MV builder compute instances depending on the latency and lag requirements to hydrate the views. From a technical stand point, this compute layer helps manage connections between partitions in a more efficient manner even when the data size is huge and the number of partitions is high.
113
110
114
111
The compute containers are shared among all MVs within the database account. Each provisioned compute container spawns off multiple tasks that read change feed from base table partitions and write data to MV (which is also another table) after transforming them as per MV definition for every MV in the database account.
115
112
116
-
### Create materialized view builder
113
+
### Create a materialized view builder
117
114
118
-
#### Using Portal
115
+
#### Using the Azure portal
119
116
120
117
1. Sign in to the [Azure portal](https://portal.azure.com/).
121
118
122
-
2. Navigate to your Azure Cosmos DB API for Cassandra account.
119
+
1. Navigate to your Azure Cosmos DB API for Cassandra account.
120
+
121
+
1. Go to the **Materialized View Builder** pane underneath the **Settings** section.
123
122
124
-
3. Go to the **Materialized View Builder** pane underneath the **Settings** section. (this will show up only when Materialized View feature is enabled as mentioned earlier in the documentation)
123
+
> [!NOTE]
124
+
> This pane will show up only when Materialized View feature is enabled as mentioned previously in the documentation.
125
125
126
-
4. Select **Materialized View Builder**.
126
+
1. Select **Materialized View Builder**.
127
127
128
-
5. Configure as required.
128
+
1. Configure the builder.
129
129
130
130
:::image type="content" source="./media/materialized-view/provision_mv_builder.png" alt-text="Provision Materialized view builder on portal for Azure Cosmos DB API for Cassandra":::
131
131
132
-
#### Using AZ CLI
132
+
#### Using Azure CLI
133
133
134
134
Please use this link to install Microsoft Azure CLI 'cosmosdb-preview' Extension: [Azure CLI extension overview](https://learn.microsoft.com/cli/azure/azure-cli-extensions-overview)
135
135
136
-
Refer to this link on documentation for AZ CLI command to create Materialized View builder: [AZ CLI documentation](https://github.com/Azure/azure-cli-extensions/tree/main/src/cosmosdb-preview#create-a-cosmosdb-materialized-views-builder-service-resource)
136
+
Refer to this link on documentation for Azure CLI command to create Materialized View builder: [Azure CLI documentation](https://github.com/Azure/azure-cli-extensions/tree/main/src/cosmosdb-preview#create-a-cosmosdb-materialized-views-builder-service-resource)
137
137
138
138
Enable Materialized View for account using command as in below example
139
139
140
-
```azurecli-interactive
140
+
```azurecli
141
141
az cosmosdb service create --resource-group "TestRG" --account-name "testaccount" --name "MaterializedViewsBuilder" --kind "MaterializedViewsBuilder" --count 1 --size "Cosmos.D4s"
142
142
```
143
143
144
-
#### Using AZ REST
144
+
#### Using the REST API and Azure CLI
145
145
146
146
Following this step, you'll also need to provision a Materialized View Builder:
147
147
148
-
```
148
+
```azurecli
149
149
az rest --method PUT --uri https://management.azure.com/subscriptions/{{subscriptionId}}/resourcegroups/{{resourceGroup}}/providers/Microsoft.DocumentDb/databaseAccounts/{{accountName}}/services/materializedViewsBuilder?api-version=2021-07-01-preview --body @body.txt
150
+
```
150
151
151
-
body.txt content:
152
+
```json
152
153
{
153
-
"properties":
154
-
{
154
+
"properties": {
155
155
"serviceType": "materializedViewsBuilder",
156
156
"instanceCount": 1,
157
157
"instanceSize": "Cosmos.D4s"
158
158
}
159
159
}
160
160
```
161
161
162
-
Wait for a couple of minutes and check the status using the below, the status in the output should have become Running:
162
+
Wait for a couple of minutes and check the status using `az rest` again, the status in the output should now be `Running`:
163
163
164
-
```
164
+
```azurecli
165
165
az rest --method GET --uri https://management.azure.com/subscriptions/{{subscriptionId}}/resourcegroups/{{resourceGroup}}/providers/Microsoft.DocumentDb/databaseAccounts/{{accountName}}/services/materializedViewsBuilder?api-version=2021-07-01-preview
166
166
```
167
167
@@ -173,11 +173,11 @@ Once your account and Materialized View Builder is set up, you should be able to
173
173
174
174
Install standalone CQLSH tool using the link [CQLSH Tool](https://learn.microsoft.com/azure/cosmos-db/cassandra/support#cql-shell)
175
175
176
-
Update Connection String
176
+
Update Connection String
177
177
178
178
Quickstart: [API for Cassandra with CQLSH - Azure Cosmos DB | Microsoft Learn](https://learn.microsoft.com/azure/cosmos-db/cassandra/manage-data-cqlsh#update-your-connection-string)
179
179
180
-
CQL commands to create MV
180
+
CQL commands to create MV
181
181
182
182
```
183
183
test@cqlsh> CREATE KEYSPACE IF NOT EXISTS uprofile
However, there are a few caveats with Cosmos DB Cassandra API’s preview implementation of Materialized Views:
225
+
223
226
- Materialized Views can't be created on a table that existed before the account was onboarded to support materialized views. Create new table after account is onboarded on which materialized views can be defined.
224
227
- For the MV definition’s WHERE clause, only “IS NOT NULL” filters are currently allowed.
225
228
- After a Materialized View is created against a base table, ALTER TABLE ADD operations aren't allowed on the base table’s schema - they're allowed only if none of the MVs have select * in their definition.
@@ -228,11 +231,11 @@ However, there are a few caveats with Cosmos DB Cassandra API’s preview implem
228
231
- Materialized View may become inconsistent with the source table for a small number of rows after automatic regional failover. Our suggestion is to rebuild the MV in this case.
229
232
- At the moment, creating MV builder instances with 32 cores is not supported. If needed, you can provision multiple instances with smaller number of cores.
230
233
231
-
In addition to the above, note the following limitations
234
+
In addition to the above, note the following limitations
232
235
233
236
### Availability zones limitations
234
237
235
-
- Materialized views can't be enabled on an account that has Availability zone enabled regions.
238
+
- Materialized views can't be enabled on an account that has Availability zone enabled regions.
236
239
- Adding a new region with Availability zone is not supported once “enableMaterializedViews” is set to true on the account.
237
240
238
241
### Periodic backup and restore limitations
@@ -248,11 +251,8 @@ Materialized views aren't automatically restored with the restore process. Custo
248
251
- Deleting base table is not allowed if at least one MV is defined on it. All the MVs must first be deleted and then the base table can be deleted.
249
252
- Defining materialized views on containers with Static columns is not allowed
250
253
251
-
252
-
253
254
## Frequently asked questions (FAQs) …
254
255
255
-
256
256
### What transformations/actions are supported?
257
257
258
258
- Specifying a partition key that is different from base table partition key.
@@ -271,28 +271,29 @@ Autoscaling for MV builder is not available right now. The MV builder instances
271
271
272
272
The proposed billing model will be to charge the customers for:
273
273
274
-
**MV Builder compute nodes**MV Builder Compute – Single tenant layer
274
+
**MV Builder compute nodes**MV Builder Compute – Single tenant layer
275
275
276
-
**Storage**The OLTP storage of the base table and MV based on existing storage meter for Containers. LogStore won't be charged.
276
+
**Storage**The OLTP storage of the base table and MV based on existing storage meter for Containers. LogStore won't be charged.
277
277
278
-
**Request Units**The provisioned RUs for base container and Materialized View.
278
+
**Request Units**The provisioned RUs for base container and Materialized View.
279
279
280
280
### What are the different SKUs that will be available?
281
+
281
282
Refer to Pricing - [Azure Cosmos DB | Microsoft Azure](https://azure.microsoft.com/pricing/details/cosmos-db/) and check instances under Dedicated Gateway
282
283
283
284
### What type of TTL support do we have?
284
285
285
-
TTL from base table rows will be applied on MV as well. Setting table level TTL on MV is not allowed.
286
+
TTL from base table rows will be applied on MV as well. Setting table level TTL on MV is not allowed.
287
+
288
+
### Initial troubleshooting if MVs aren't up to date:
286
289
287
-
### Initial troubleshooting if MVs aren't up to date:
288
290
- Check if MV builder instances are provisioned
289
291
- Check if enough RUs are provisioned on the base table
290
292
- Check for unavailability on Base table or MV
291
293
292
294
### What type of monitoring is available in addition to the existing monitoring for Cassandra API?
293
295
294
-
295
-
- Max Materialized View Catchup Gap in Minutes – Value(t) indicates rows written to base table in last ‘t’ minutes is yet to be propagated to MV.
296
+
- Max Materialized View Catchup Gap in Minutes – Value(t) indicates rows written to base table in last ‘t’ minutes is yet to be propagated to MV.
296
297
- Metrics related to RUs consumed on base table for MV build (read change feed cost)
297
298
- Metrics related to RUs consumed on MV for MV build (write cost)
298
299
- Metrics related to resource consumption on MV builders (CPU, memory usage metrics)
@@ -305,13 +306,13 @@ TTL from base table rows will be applied on MV as well. Setting table level TTL
305
306
306
307
:::image type="content" source="./media/materialized-view/monitoring4.png" alt-text="Max CPU Metrics Screenshot for Materialized view feature on portal Azure Cosmos DB API for Cassandra":::
307
308
308
-
309
309
### What are the restore options available for MVs?
310
+
310
311
MVs can't be restored. Hence, MVs will need to be recreated once the base table is restored.
311
312
312
313
### Can you create more than one view on a base table?
313
314
314
-
Multiple views can be created on the same base table. Limit of five views is enforced.
315
+
Multiple views can be created on the same base table. Limit of five views is enforced.
315
316
316
317
### How is uniqueness enforced on the materialized view? How will the mapping between the records in base table to the records in materialized view look like?
317
318
@@ -327,12 +328,13 @@ No. Materialized Views can't be created on a table that existed before the accou
327
328
328
329
### What are the conditions on which records won't make it to MV and how to identify such records?
329
330
330
-
Below are some of the identified cases where data from base table can't be written to MV as they violate some constraints on MV table-
331
+
Below are some of the identified cases where data from base table can't be written to MV as they violate some constraints on MV table:
332
+
331
333
- Rows that don’t satisfy partition key size limit in the materialized views (2kb limit)
332
334
- Rows that don't satisfy clustering key size limit in materialized views (1kb limit)
333
-
335
+
334
336
Currently we drop these rows but plan to expose details related to dropped rows in future so that the user can reconcile the missing data.
335
337
336
338
### Materialized View with Customer Managed Keys
337
339
338
-
To create an account with support for customer managed keys and materialized views please reach out to Cosmos DB support team.
340
+
To create an account with support for customer managed keys and materialized views please reach out to Cosmos DB support team.
0 commit comments