You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/azure-monitor/platform/customer-managed-keys.md
+29-45Lines changed: 29 additions & 45 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,7 +5,7 @@ ms.subservice: logs
5
5
ms.topic: conceptual
6
6
author: yossi-y
7
7
ms.author: yossiy
8
-
ms.date: 05/07/2020
8
+
ms.date: 05/13/2020
9
9
10
10
---
11
11
# Azure Monitor customer-managed key
@@ -14,53 +14,37 @@ This article provides background information and steps to configure customer-Man
14
14
15
15
We recommend you review [Limitations and constraints](#limitations-and-constraints) below before configuration.
16
16
17
-
## Disclaimers
18
17
19
-
- The CMK capability is delivered on a dedicated Log Analytics cluster, which is a physical cluster and a data store that is suitable for customers sending 1TB per day or more
18
+
## Customer-managed key (CMK) overview
20
19
21
-
- The CMK pricing model isn't available currently and it isn't covered in this article. A pricing model for dedicated Log Analytics cluster is expected in the second quarter of calendar year (CY) 2020 and will apply to any existing CMK deployments.
20
+
Encryption at Rest(https://docs.microsoft.com/azure/security/fundamentals/encryption-atrest) is a common privacy and security requirement in organizations. You can let Azure completely manage Encryption at Rest, while you have various options to closely manage encryption or encryption keys.
22
21
23
-
## Customer-managed key (CMK) overview
22
+
Azure Monitor ensures that all data is encrypted at rest using Azure-managed keys. Azure Monitor also provides an option for data encryption using your own key that is stored in your [Azure Key Vault](https://docs.microsoft.com/azure/key-vault/key-vault-overview) and accessed by Storage using system-assigned [managed identity](https://docs.microsoft.com/azure/active-directory/managed-identities-azure-resources/overview) authentication. This key can be either [software or hardware-HSM protected](https://docs.microsoft.com/azure/key-vault/key-vault-overview).
24
23
25
-
[Encryption at Rest](https://docs.microsoft.com/azure/security/fundamentals/encryption-atrest)
26
-
is a common privacy and security requirement in organizations. You can
27
-
let Azure completely manage Encryption at Rest, while you have various
28
-
options to closely manage encryption or encryption keys.
24
+
Azure Monitor use of encryption is identical to the way [Azure Storage encryption](https://docs.microsoft.com/azure/storage/common/storage-service-encryption#about-azure-storage-encryption) operates.
29
25
30
-
Azure Monitor storage ensures that all data encrypted at
31
-
rest using Azure-managed keys while stored in Azure Storage. Azure Monitor also
32
-
provides an option for data encryption using your own key that is stored
33
-
in your [Azure Key Vault](https://docs.microsoft.com/azure/key-vault/key-vault-overview),
34
-
which is accessed using system-assigned [managed identity](https://docs.microsoft.com/azure/active-directory/managed-identities-azure-resources/overview) authentication. This key can be either [software or hardware-HSM
The CMK capability is delivered on dedicated Log Analytics clusters. The [Log Analytics clusters pricing model](https://docs.microsoft.com/en-us/azure/azure-monitor/platform/manage-cost-storage#log-analytics-dedicated-clusters) uses Capacity Reservations starting at a 1000 GB/day level.
39
27
40
-
The frequency that Azure Monitor Storage accesses Key Vault for wrap and
41
-
unwrap operations is between 6 to 60 seconds. Azure Monitor Storage always respects changes in key permissions within an hour.
28
+
Data ingested in the last 14 days is also kept in hot-cache (SSD-backed) for efficient query engine operation. This data remains encrypted with Microsoft keys regardless CMK configuration, but your control over SSD data adheres to [key revocation](https://docs.microsoft.com/en-us/azure/azure-monitor/platform/customer-managed-keys#cmk-kek-revocation). We are working to have SSD data encrypted with CMK in the second half of 2020.
42
29
43
-
Ingested data in last 14 days is also kept in hot-cache (SSD-backed) for efficient query engine operation. This data remains encrypted with Microsoft keys regardless CMK configuration, but your control over SSD data adheres to a [key revocation](#cmk-kek-revocation) and is inaccessible. We are working to have the SSD data encrypted with CMK in the second half of 2020.
30
+
The frequency that Azure Monitor Storage accesses Key Vault for wrap and unwrap operations is between 6 to 60 seconds. Azure Monitor Storage always respects changes in key permissions within an hour.
44
31
45
32
## How CMK works in Azure Monitor
46
33
47
34
Azure Monitor leverages system-assigned managed identity to grant access
48
35
to your Azure Key Vault. System-assigned managed identity can only be
49
-
associated with a single Azure resource. The identity of the dedicated Log Analytics cluster is supported at the cluster level and this
50
-
dictates that the CMK capability is delivered on dedicated Log Analytics cluster. To support CMK on multiple workspaces, a new Log Analytics
36
+
associated with a single Azure resource. The identity of the Log Analytics cluster is supported at the cluster level and this
37
+
dictates that the CMK capability is delivered on a dedicated Log Analytics cluster. To support CMK on multiple workspaces, a new Log Analytics
51
38
*Cluster* resource performs as an intermediate identity connection
52
-
between your Key Vault and your Log Analytics workspaces. This concept
53
-
maintains the identity between the dedicated Log Analytics cluster and the Log Analytics *Cluster*
54
-
resource, while the data of associated workspaces is protected
55
-
with your Key Vault key. The dedicated Log Analytics cluster storage uses the
39
+
between your Key Vault and your Log Analytics workspaces, which maintains the identity between the Log Analytics cluster and your Key Vault. The Log Analytics cluster storage uses the
56
40
managed identity that\'s associated with the *Cluster* resource to
57
41
authenticate and access your Azure Key Vault via Azure Active Directory.
2.Customer's Log Analytics *Cluster* resource having managed identity with permissions to Key Vault – The identity is supported at the dedicated Log Analytics cluster level.
62
-
3. Dedicated Log Analytics cluster.
63
-
4.Customer's workspaces associated to *Cluster* resource for CMK encryption.
44
+
1. Key Vault
45
+
2. Log Analytics *Cluster* resource having managed identity with permissions to Key Vault -- The identity is propagated to the underlay dedicated Log Analytics cluster storage
46
+
3. Dedicated Log Analytics cluster
47
+
4.Workspaces associated to *Cluster* resource for CMK encryption
64
48
65
49
## Encryption keys operation
66
50
@@ -72,7 +56,7 @@ There are 3 types of keys involved in Storage data encryption:
72
56
73
57
The following rules apply:
74
58
75
-
- The dedicated Log Analytics cluster storage accounts generate unique encryption key for every Storage account, which is known as the AEK.
59
+
- The Log Analytics cluster storage accounts generate unique encryption key for every Storage account, which is known as the AEK.
76
60
77
61
- The AEK is used to derive DEKs, which are the keys that are used to
78
62
encrypt each block of data written to disk.
@@ -88,7 +72,7 @@ The following rules apply:
88
72
89
73
## CMK provisioning procedure
90
74
91
-
1. Subscription whitelisting -- To assure that we have the required capacity in your region for dedicated Log Analytics cluster, we need to verify and whitelist your subscription beforehand
75
+
1. Subscription whitelisting -- To assure that we have the required capacity in your region to provision a Log Analytics cluster, we need to verify and whitelist your subscription beforehand
92
76
2. Creating Azure Key Vault and storing key
93
77
3. Creating a *Cluster* resource
94
78
5. Granting permissions to your Key Vault
@@ -227,7 +211,7 @@ The identity is assigned to the *Cluster* resource at creation time.
227
211
228
212
200 OK and header.
229
213
230
-
While it takes the provisioning of the dedicated Log Analytics cluster a while to complete, you can check the provisioning state in two ways:
214
+
While it takes the provisioning of the Log Analytics cluster a while to complete, you can check the provisioning state in two ways:
231
215
232
216
1. Copy the Azure-AsyncOperation URL value from the response and follow the [asynchronous operations status check](#asynchronous-operations-and-status-check).
233
217
2. Send a GET request on the *Cluster* resource and look at the *provisioningState* value. It is *ProvisioningAccount* while provisioning and *Succeeded* when completed.
@@ -372,7 +356,7 @@ You need to have 'write' permissions to both your workspace and *Cluster* resour
372
356
- In *Cluster* resource: Microsoft.OperationalInsights/clusters/write
373
357
374
358
> [!IMPORTANT]
375
-
> This step should be performed only after the completion of the dedicated Log Analytics cluster provisioning. If you associate workspaces and ingest data prior to the provisioning, ingested data will be dropped and won't be recoverable.
359
+
> This step should be performed only after the completion of the Log Analytics cluster provisioning. If you associate workspaces and ingest data prior to the provisioning, ingested data will be dropped and won't be recoverable.
376
360
377
361
**Associate a workspace**
378
362
@@ -437,27 +421,27 @@ GET https://management.azure.com/subscriptions/<subscription-id>/resourcegroups/
437
421
438
422
## CMK (KEK) revocation
439
423
440
-
You can revoke access to data by disabling your key, or deleting the *Cluster* resource access policy in your Key Vault. The dedicated Log Analytics cluster storage will always respect changes in key permissions within an hour or sooner, and Storage will become unavailable. Any data ingested to workspaces associated with your*Cluster*resource gets dropped and queries will fail. Previously ingested data remains inaccessible in storage as while your *Cluster* resource and your workspaces aren't deleted. Inaccessible data is governed by the data-retention policy and will be purged when retention is reached.
424
+
You can revoke access to data by disabling your key, or deleting the *Cluster* resource access policy in your Key Vault. The Log Analytics cluster storage will always respect changes in key permissions within an hour or sooner and Storage will become unavailable. Any new data ingested to workspaces associated with your*Cluster*resource gets dropped and won't be recoverable, data is inaccessible and queries to these workspaces fail. Previously ingested data remains in storage as long as your *Cluster* resource and your workspaces aren't deleted. Inaccessible data is governed by the data-retention policy and will be purged when retention is reached.
441
425
442
426
Ingested data in last 14 days is also kept in hot-cache (SSD-backed) for efficient query engine operation. This gets deleted on key revocation operation and becomes inaccessible as well.
443
427
444
428
Storage periodically polls your Key Vault to attempt to unwrap the encryption key and once accessed, data ingestion and query resume within 30 minutes.
445
429
446
430
## CMK (KEK) rotation
447
431
448
-
Rotation of CMK requires explicit update to the *Cluster* resource with the new key version in Azure Key Vault. Follow the instructions in "Update *Cluster* resource with Key identifier details" step. If you don't update the new key identifier details in the *Cluster* resource, the dedicated Log Analytics cluster storage will keep using your previous key.
432
+
Rotation of CMK requires explicit update to the *Cluster* resource with the new key version in Azure Key Vault. Follow the instructions in "Update *Cluster* resource with Key identifier details" step. If you don't update the new key identifier details in the *Cluster* resource, the Log Analytics cluster storage will keep using your previous key for encryption. If you disable or delete your old key before updating the new key in the *Cluster* resource, you will get into [key revocation](#cmk-kek-revocation) state.
449
433
450
434
All your data remains accessible after the key rotation operation including data ingested before the rotation and after it, since data always encrypted with Account Encryption Key (AEK) while AEK is now being encrypted with your new Key Encryption Key (KEK) version in Key Vault.
451
435
452
436
## Limitations and constraints
453
437
454
-
- The CMK is supported on dedicated Log Analytics cluster suitable for customers sending 1TB per day or more.
438
+
- The CMK is supported on dedicated Log Analytics cluster and suitable for customers sending 1TB per day or more.
455
439
456
440
- The max number of *Cluster* resources per region and subscription is 2
457
441
458
-
- You can associate a workspace to your *Cluster* resource and then de-associate it when CMK for its data is no longer needed or any other reason. The number of workspace association that you can perform on a workspace in a period of 30 days is limited to 2
442
+
- You can associate a workspace to your *Cluster* resource and then disassociate it when CMK for its data is no longer needed or any other reason. The number of workspace association that you can perform on a workspace in a period of 30 days is limited to 2
459
443
460
-
- Workspace association to *Cluster* resource should be carried ONLY after you have verified that the dedicated Log Analytics cluster provisioning was completed. Data sent to your workspace prior to the completion will be dropped and won't be recoverable.
444
+
- Workspace association to *Cluster* resource should be carried ONLY after you have verified that the Log Analytics cluster provisioning was completed. Data sent to your workspace prior to the completion will be dropped and won't be recoverable.
461
445
462
446
- CMK encryption applies to newly ingested data after the CMK
463
447
configuration. Data that was ingested prior to the CMK
@@ -564,9 +548,9 @@ All your data remains accessible after the key rotation operation including data
564
548
}
565
549
```
566
550
567
-
-**De-associate workspace**
551
+
-**Disassociate workspace**
568
552
569
-
You need 'write' permissions on the workspace and *Cluster* resource to perform this operation. You can de-associate a workspace from your *Cluster* resource at any time. New ingested data after the de-association operation is stored in Log Analytics storage and encrypted with Microsoft key. You can query you data that was ingested to your workspace before and after the de-association seamlessly as long as the *Cluster* resource is provisioned and configured with valid Key Vault key.
553
+
You need 'write' permissions on the workspace and *Cluster* resource to perform this operation. You can disassociate a workspace from your *Cluster* resource at any time. New ingested data after the de-association operation is stored in Log Analytics storage and encrypted with Microsoft key. You can query you data that was ingested to your workspace before and after the de-association seamlessly as long as the *Cluster* resource is provisioned and configured with valid Key Vault key.
570
554
571
555
This Resource Manager request is asynchronous operation.
572
556
@@ -581,12 +565,12 @@ All your data remains accessible after the key rotation operation including data
581
565
Ingested data after the de-association operation is stored in Log Analytics storage, this can take 90 minutes to complete. You can check the workspace de-association state in two ways:
582
566
583
567
1. Copy the Azure-AsyncOperation URL value from the response and follow the [asynchronous operations status check](#asynchronous-operations-and-status-check).
584
-
2. Send a [Workspaces – Get](https://docs.microsoft.com/rest/api/loganalytics/workspaces/get) request and observe the response, de-associated workspace will won't have the *clusterResourceId* under *features*.
568
+
2. Send a [Workspaces – Get](https://docs.microsoft.com/rest/api/loganalytics/workspaces/get) request and observe the response, disassociated workspace won't have the *clusterResourceId* under *features*.
585
569
586
570
587
571
-**Delete your *Cluster* resource**
588
572
589
-
You need 'write' permissions on the *Cluster* resource to perform this operation. A soft-delete operation is performed to allow the recovery of your *Cluster* resource including its data within 14 days, whether the deletion was accidental or intentional. The *Cluster* resource name remains reserved during the soft-delete period and you can't create a new cluster with that name. After the soft-delete period, the *Cluster* resource name is released, your *Cluster* resource and data are permanently deleted and are non-recoverable. Any associated workspace gets de-associated from the *Cluster* resource on delete operation. New ingested data is stored in Log Analytics storage and encrypted with Microsoft key. The workspaces de-associated operation is asynchronous and can take up to 90 minutes to complete.
573
+
You need 'write' permissions on the *Cluster* resource to perform this operation. A soft-delete operation is performed to allow the recovery of your *Cluster* resource including its data within 14 days, whether the deletion was accidental or intentional. The *Cluster* resource name remains reserved during the soft-delete period and you can't create a new cluster with that name. After the soft-delete period, the *Cluster* resource name is released, your *Cluster* resource and data are permanently deleted and are non-recoverable. Any associated workspace gets disassociated from the *Cluster* resource on delete operation. New ingested data is stored in Log Analytics storage and encrypted with Microsoft key. The workspaces disassociated operation is asynchronous and can take up to 90 minutes to complete.
@@ -599,7 +583,7 @@ All your data remains accessible after the key rotation operation including data
599
583
600
584
-**Recover your *Cluster* resource and your data**
601
585
602
-
A *Cluster* resource that was deleted in the last 14 days is in soft-delete state and can be recovered with its data. Since all workspaces got de-associated from the *Cluster* resource on deletion, you need to re-associate your workspaces after the recovery for CMK encryption. The recovery operation is performed manually by the product group currently. Use your Microsoft channel for recovery requests.
586
+
A *Cluster* resource that was deleted in the last 14 days is in soft-delete state and can be recovered with its data. Since all workspaces got disassociated from the *Cluster* resource with *Cluster* resource deletion, you need to re-associate your workspaces after the recovery for CMK encryption. The recovery operation is performed manually by the product group currently. Use your Microsoft channel for recovery requests.
0 commit comments