You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
description: Information and steps to configure Customer-Managed Key (CMK) to encrypt data in your Log Analytics workspaces using an Azure Key Vault key.
This article provides background information and steps to configure Customer-Managed Keys (CMK) for your Log Analytics workspaces and Application Insights components. Once configured, any data sent to your workspaces or components is encrypted with your Azure Key Vault key.
13
+
This article provides background information and steps to configure customer-Managed Keys (CMK) for your Log Analytics workspaces. Once configured, any data sent to your workspaces is encrypted with your Azure Key Vault key.
14
14
15
15
We recommend you review [Limitations and constraints](#limitations-and-constraints) below before configuration.
16
16
@@ -24,12 +24,6 @@ We recommend you review [Limitations and constraints](#limitations-and-constrain
24
24
25
25
- The CMK pricing model isn't available currently and it isn't covered in this article. A pricing model for dedicated ADX cluster is expected in the second quarter of calendar year (CY) 2020 and will apply to any existing CMK deployments.
26
26
27
-
- This article describes the CMK configuration for Log Analytics workspaces. CMK for Application Insights components is also supported using this article while differences are listed in the Appendix.
28
-
29
-
> [!NOTE]
30
-
> Log Analytics and Application Insights are using the same data-store platform and query engine.
31
-
> We are bringing these two stores together via integration of Application Insights into Log Analytics to create a single unified logs store under Azure Monitor. This change is planned for the second quarter of calendar year 2020. If you don't have to deploy CMK for your Application Insights data by then, we recommend waiting for the completion of the consolidation since such deployments will be disrupted by the consolidation and you will have to re-configure CMK after the migration to Log Analytics workspace. The 1TB per day minimum applies at the cluster level and until the consolidation completes during second quarter, Application Insights and Log Analytics require separate clusters.
32
-
33
27
## Customer-managed key (CMK) overview
34
28
35
29
[Encryption at Rest](https://docs.microsoft.com/azure/security/fundamentals/encryption-atrest)
@@ -75,7 +69,7 @@ authenticate and access your Azure Key Vault via Azure Active Directory.
75
69
3. Azure Monitor dedicated ADX cluster.
76
70
4. Customer's workspaces associated to *Cluster* resource for CMK encryption.
77
71
78
-
## Encryption keys management
72
+
## Encryption keys operation
79
73
80
74
There are 3 types of keys involved in Storage data encryption:
81
75
@@ -101,8 +95,6 @@ The following rules apply:
101
95
102
96
## CMK provisioning procedure
103
97
104
-
For Application Insights CMK configuration, follow the Appendix content for steps 3 and 6.
105
-
106
98
1. Subscription whitelisting -- this is required for this early access
107
99
feature
108
100
2. Creating Azure Key Vault and storing key
@@ -204,9 +196,7 @@ These settings are available via CLI and PowerShell:
204
196
205
197
### Create *Cluster* resource
206
198
207
-
This resource is used as an intermediate identity connection between your Key Vault and your Log Analytics workspaces. After you receive confirmation that your subscriptions were whitelisted, create a Log Analytics *Cluster* resource at the region where your workspaces are located. Application Insights and Log Analytics require separate *Cluster* resources types. The type of the *Cluster* resource is defined at creation time by setting the *clusterType* property to either *LogAnalytics*, or *ApplicationInsights*. The Cluster resource type can't be altered after.
208
-
209
-
For Application Insights CMK configuration, follow the Appendix content.
199
+
This resource is used as an intermediate identity connection between your Key Vault and your Log Analytics workspaces. After you receive confirmation that your subscriptions were whitelisted, create a Log Analytics *Cluster* resource at the region where your workspaces are located.
210
200
211
201
You must specify the capacity reservation level (sku) when creating a *Cluster* resource. The capacity reservation level can be in the range of 1,000 to 2,000 GB per day and you can update it in steps of 100 later. If you need capacity reservation level higher than 2,000 GB per day, reach your Microsoft contact to enable it. This property doesn't affect billing currently -- once pricing model for dedicated cluster is introduced, billing will apply to any existing CMK deployments.
212
202
@@ -215,7 +205,7 @@ You must specify the capacity reservation level (sku) when creating a *Cluster*
215
205
This Resource Manager request is asynchronous operation.
216
206
217
207
```rst
218
-
PUT https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.OperationalInsights/clusters/<cluster-name>?api-version=2019-08-01-preview
208
+
PUT https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.OperationalInsights/clusters/<cluster-name>?api-version=2020-03-01-preview
219
209
Authorization: Bearer <token>
220
210
Content-type: application/json
221
211
@@ -247,7 +237,7 @@ During the early access period of the feature, the ADX cluster is provisioned ma
247
237
During the early access period of the feature, the ADX cluster is provisioned manually by the product team once the previous steps are completed. Use your Microsoft channel for this step and provide the *Cluster* resource response.
248
238
249
239
```rst
250
-
GET https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.OperationalInsights/clusters/<cluster-name>?api-version=2019-08-01-preview
240
+
GET https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.OperationalInsights/clusters/<cluster-name>?api-version=2020-03-01-preview
251
241
Authorization: Bearer <token>
252
242
```
253
243
@@ -375,7 +365,6 @@ A response to GET request on the *Cluster* resource should look like this when K
375
365
```
376
366
377
367
### Workspace association to *Cluster* resource
378
-
For Application Insights CMK configuration, follow the Appendix content for this step.
379
368
380
369
You need to have 'write' permissions to both your workspace and *Cluster* resource to perform this operation, which include these actions:
381
370
@@ -477,9 +466,7 @@ All your data is accessible after the key rotation operation including data inge
-[Purge protection](https://docs.microsoft.com/azure/key-vault/key-vault-ovw-soft-delete#purge-protection) should be turned on to guard against force deletion of the secret / vault even after soft delete
481
-
482
-
- Application Insights and Log Analytics require separate *Cluster* resources. The type of the *Cluster* resource is defined at creation time by setting the "clusterType" property to either 'LogAnalytics', or 'ApplicationInsights'. The *Cluster* resource type can't be altered.
469
+
-[Purge protection](https://docs.microsoft.com/azure/key-vault/key-vault-ovw-soft-delete#purge-protection) should be turned on to guard against force deletion of the secret / vault even after soft delete.
483
470
484
471
-*Cluster* resource move to another resource group or subscription
485
472
isn't supported currently.
@@ -489,9 +476,10 @@ All your data is accessible after the key rotation operation including data inge
489
476
- Workspace association to *Cluster* resource will fail if it is
490
477
associated to another *Cluster* resource
491
478
479
+
492
480
## Troubleshooting and management
493
481
494
-
- Key Vault availability
482
+
- Key Vault availability considerations
495
483
- In normal operation -- Storage caches AEK for short periods of time and goes back to Key Vault to unwrap periodically.
496
484
497
485
- Transient connection errors -- Storage handles transient errors (timeouts, connection failures, DNS issues) by allowing keys to stay in cache for a short while longer and this overcomes any small blips in availability. The query and ingestion capabilities continue without interruption.
@@ -507,6 +495,8 @@ All your data is accessible after the key rotation operation including data inge
507
495
508
496
- If you get conflict error when creating a *Cluster* resource – It may be that you have deleted your *Cluster* resource in the last 14 days and it’s in a soft-delete period. The *Cluster* resource name remains reserved during the soft-delete period and you can't create a new cluster with that name. The name is released after the soft-delete period when the *Cluster* resource is permanently deleted.
509
497
498
+
- If you update your *Cluster* resource while an operation is in progress, the operation will fail.
499
+
510
500
- Get all *Cluster* resources for a resource group:
511
501
512
502
```rst
@@ -531,10 +521,10 @@ All your data is accessible after the key rotation operation including data inge
@@ -562,7 +552,7 @@ All your data is accessible after the key rotation operation including data inge
562
552
563
553
- Update *capacity reservation* in *Cluster* resource -- when the data volume to your associated workspaces change and you want to update the capacity reservation level for billing considerations, follow the [update *Cluster* resource](#update-cluster-resource-with-key-identifier-details) and provide your new capacity value. The capacity reservation level can be in the range of 1,000 to 2,000 GB per day and in steps of 100. For level higher than 2,000 GB per day, reach your Microsoft contact to enable it.
564
554
565
-
- Delete your *Cluster* resource -- a soft-delete operation is performed to allow the recovery of your *Cluster* resource including its data within 14 days, whether the deletion was accidental or intentional. The *Cluster* resource name remains reserved during the soft-delete period and you can't create a new cluster with that name. After the soft-delete period, The *Cluster* resource name is released, your *Cluster* resource and data are permanently deleted and are non-recoverable. Any associated workspace gets de-associated from the *Cluster* resource on delete operation. New ingested data is stored in shared Log Analytics storage and encrypted with Microsoft key. The workspaces de-associated operation is asynchronous.
555
+
- Delete your *Cluster* resource -- You need 'write' permissions on the *Cluster* resource to perform this operation. A soft-delete operation is performed to allow the recovery of your *Cluster* resource including its data within 14 days, whether the deletion was accidental or intentional. The *Cluster* resource name remains reserved during the soft-delete period and you can't create a new cluster with that name. After the soft-delete period, The *Cluster* resource name is released, your *Cluster* resource and data are permanently deleted and are non-recoverable. Any associated workspace gets de-associated from the *Cluster* resource on delete operation. New ingested data is stored in shared Log Analytics storage and encrypted with Microsoft key. The workspaces de-associated operation is asynchronous.
@@ -576,165 +566,3 @@ All your data is accessible after the key rotation operation including data inge
576
566
- Recover your *Cluster* resource and your data --
577
567
A *Cluster* resource that was deleted in the last 14 days is in soft-delete state and can be recovered. This is performed manually by the product group currently. Use your Microsoft channel for recovery requests.
578
568
579
-
## Appendix
580
-
581
-
Application Insights Customer Managed Key (CMK) is supported as well, though you should consider the following change to help you plan the deployment of CMK for your Application Insight components.
582
-
583
-
Log Analytics and Application Insights are using the same data-store
584
-
platform and query engine. We are bringing these two stores together
585
-
via integration of Application Insights into Log Analytics to provide a
586
-
single unified logs store under Azure Monitor by the second quarter of
587
-
2020. This change will bring your Application Insight data into Log
588
-
Analytics workspaces and make queries, insights, and other improvements
589
-
possible while the configuration of CMK on your workspace, will also
590
-
apply to your Application Insights data.
591
-
592
-
> [!NOTE]
593
-
> If you don't have to deploy CMK for your Application Insight data before the integration, we recommend waiting with Application Insights CMK since such deployments will be disrupted by the integration and you will have to re-configure CMK after the migration to Log Analytics workspace. The 1TB per day minimum applies at the cluster level and until the consolidation completes during second quarter, Application Insights and Log Analytics require separate clusters.
594
-
595
-
## Application Insights CMK configuration
596
-
597
-
The configuration of Application Insights CMK is identical to the process illustrated in this article, including constraints and troubleshooting except these steps:
598
-
599
-
- Create a *Cluster* resource
600
-
- Associate a component to a *Cluster* resource
601
-
602
-
When configuring CMK for Application Insights, use these steps instead
603
-
of the ones listed above.
604
-
605
-
### Create a *Cluster* resource
606
-
607
-
This resource is used as intermediate identity connection between your Key Vault and your components. AFTER you received a confirmation that your subscriptions were whitelisted, create a Log Analytics *Cluster* resource at the region where your components are located. The type of the *Cluster* resource is defined at creation time by setting the *clusterType* property to either *LogAnalytics*, or *ApplicationInsights*. It should be *ApplicationInsights* for Application Insights CMK. The *clusterType* setting can't be altered after the configuration.
608
-
609
-
**Create**
610
-
611
-
This Resource Manager request is asynchronous operation.
612
-
613
-
```rst
614
-
PUT https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.OperationalInsights/clusters/<cluster-name>?api-version=2020-03-01-preview
615
-
Authorization: Bearer <token>
616
-
Content-type: application/json
617
-
618
-
{
619
-
"identity": {
620
-
"type": "systemAssigned"
621
-
},
622
-
"sku": {
623
-
"name": "capacityReservation",
624
-
"Capacity": 1000
625
-
},
626
-
"properties": {
627
-
"clusterType":"ApplicationInsights"
628
-
},
629
-
"location": "<region-name>"
630
-
}
631
-
```
632
-
633
-
**Response**
634
-
635
-
200 OK and header.
636
-
During the early access period of the feature, the ADX cluster is provisioned manually. While it takes the provisioning of the underly ADX cluster a while to complete, you can check the provisioning state in two ways:
637
-
1. Copy the Azure-AsyncOperation URL value from the response and follow the [asynchronous operations status check](#asynchronous-operations-and-status-check).
638
-
2. Send a GET request on the *Cluster* resource and look at the *provisioningState* value. It is *ProvisioningAccount* while provisioning and *Succeeded* when completed.
639
-
640
-
### Associate a component to a *Cluster* resource using [Components - Create Or Update](https://docs.microsoft.com/rest/api/application-insights/components/createorupdate) API
641
-
642
-
You need to have 'write' permissions on both your component and *Cluster* resource to perform this operation, which include these actions:
643
-
644
-
- In component: Microsoft.Insights/component/write
645
-
- In *Cluster* resource: Microsoft.OperationalInsights/clusters/write
646
-
647
-
> [!IMPORTANT]
648
-
> This step should be performed only after ADX cluster provisioning. If you associate components and ingest data prior to the provisioning, ingested data will be dropped and won't be recoverable.
649
-
> To verify that the ADX cluster is provisioned, execute *Cluster* resource Get REST API and check that the *provisioningState* value is *Succeeded*.
650
-
651
-
```rst
652
-
GET https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.OperationalInsights/clusters/<cluster-name>?api-version=2020-03-01-preview
> Copy and keep the response since you will need it in next steps.
688
-
689
-
**Associate a component**
690
-
691
-
```rst
692
-
PUT https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Insights/components/<component-name>?api-version=2015-05-01
693
-
Authorization: Bearer <token>
694
-
Content-type: application/json
695
-
696
-
{
697
-
"properties": {
698
-
"clusterDefinitionId": "cluster-id"
699
-
},
700
-
"location": "<region-name>",
701
-
"kind": "<component-type>"
702
-
}
703
-
```
704
-
"clusterDefinitionId" is the "clusterId" value provided in the respond from the previous step.
0 commit comments