You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/azure-monitor/platform/customer-managed-keys.md
+23-27Lines changed: 23 additions & 27 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -40,7 +40,7 @@ options to closely manage encryption or encryption keys.
40
40
The Azure Monitor data-store ensures that all data encrypted at
41
41
rest using Azure-managed keys while stored in Azure Storage. Azure Monitor also
42
42
provides an option for data encryption using your own key that is stored
43
-
in [Azure Key Vault](https://docs.microsoft.com/azure/key-vault/key-vault-overview),
43
+
in your [Azure Key Vault](https://docs.microsoft.com/azure/key-vault/key-vault-overview),
44
44
which is accessed using system-assigned [managed identity](https://docs.microsoft.com/azure/active-directory/managed-identities-azure-resources/overview) authentication. This key can be either [software or hardware-HSM
The Azure Monitor use of encryption is identical to the way
@@ -50,7 +50,7 @@ operates.
50
50
The frequency that Azure Monitor Storage accesses Key Vault for wrap and
51
51
unwrap operations is between 6 to 60 seconds. Azure Monitor Storage always respects changes in key permissions within an hour.
52
52
53
-
Ingested data in last 14 days is also kept in hot-cache (SSD-backed) for efficient query engine operation. This data remains encrypted with Microsoft keys regardless CMK configuration, but we are working to have the SSD encrypted with CMK early 2020.
53
+
Ingested data in last 14 days is also kept in hot-cache (SSD-backed) for efficient query engine operation. This data remains encrypted with Microsoft keys regardless CMK configuration, but we are working to have the SSD encrypted with CMK in the first half of 2020.
54
54
55
55
## How CMK works in Azure Monitor
56
56
@@ -73,19 +73,19 @@ authenticate and access your Azure Key Vault via Azure Active Directory.
73
73
1. Customer’s Key Vault.
74
74
2. Customer’s Log Analytics *Cluster* resource having managed identity with permissions to Key Vault – The identity is supported at the data-store (ADX cluster) level.
75
75
3. Azure Monitor dedicated ADX cluster.
76
-
4. Customer’s workspaces associated to Cluster resource for CMK encryption.
76
+
4. Customer’s workspaces associated to *Cluster* resource for CMK encryption.
77
77
78
78
## Encryption keys management
79
79
80
80
There are 3 types of keys involved in Storage data encryption:
81
81
82
-
-**KEK** - Key Encryption Key in Key Vault (CMK)
82
+
-**KEK** - Key Encryption Key (CMK)
83
83
-**AEK** - Account Encryption Key
84
84
-**DEK** - Data Encryption Key
85
85
86
86
The following rules apply:
87
87
88
-
- The ADX storage account generates a unique encryption key for every storage account, which is known as the AEK.
88
+
- The ADX storage accounts generate unique encryption key for every Storage account, which is known as the AEK.
89
89
90
90
- The AEK is used to derive DEKs, which are the keys that are used to
91
91
encrypt each block of data written to disk.
@@ -129,12 +129,10 @@ Where *eyJ0eXAiO....* represents the full Authorization token.
129
129
You can acquire the token using one of these methods:
130
130
131
131
1. Use [App registrations](https://docs.microsoft.com/graph/auth/auth-concepts#access-tokens) method.
132
-
133
132
2. In the Azure portal
134
133
1. Navigate to Azure portal in "developer tool (F12)
135
134
1. Look for authorization string under "Request Headers" in one of the "batch?api-version" instances. It looks like: "authorization: Bearer \<token\>".
136
135
1. Copy and add it to your API call per the examples below.
137
-
138
136
3. Navigate to Azure REST documentation site. Press "Try it" on any API and copy the Bearer token.
139
137
140
138
### Subscription whitelisting
@@ -146,19 +144,19 @@ CMK capability is an early access feature. The subscriptions where you plan to c
146
144
147
145
### Storing encryption key (KEK)
148
146
149
-
Create or use an Azure Key Vault that you already have to generate, or import a key to be used for data encryption. The Azure Key Vault must be configured as recoverable to protect your key and the access to your data in Azure Monitor. You can verify this configuration under properties in your in your Key Vault, both *Soft delete* and *Purge protection* should be enabled.
147
+
Create or use an Azure Key Vault that you already have to generate, or import a key to be used for data encryption. The Azure Key Vault must be configured as recoverable to protect your key and the access to your data in Azure Monitor. You can verify this configuration under properties in your Key Vault, both *Soft delete* and *Purge protection* should be enabled.
150
148
151
149
These settings are available via CLI and PowerShell:
-[Purge protection](https://docs.microsoft.com/azure/key-vault/key-vault-ovw-soft-delete#purge-protection) guards against force deletion of the secret / vault even after soft delete
154
152
155
153
### Create *Cluster* resource
156
154
157
-
This resource is used as an intermediate identity connection between your Key Vault and your workspaces. After you receive confirmation that your subscriptions were whitelisted, create a Log Analytics *Cluster* resource at the region where your workspaces are located. Application Insights and Log Analytics require separate *Cluster* resources types. The type of the *Cluster* resource is defined at creation time by setting the "clusterType" property to either "LogAnalytics", or "ApplicationInsights". The Cluster resource type can’t be altered after.
155
+
This resource is used as an intermediate identity connection between your Key Vault and your Log Analytics workspaces. After you receive confirmation that your subscriptions were whitelisted, create a Log Analytics *Cluster* resource at the region where your workspaces are located. Application Insights and Log Analytics require separate *Cluster* resources types. The type of the *Cluster* resource is defined at creation time by setting the *clusterType* property to either *LogAnalytics*, or *ApplicationInsights*. The Cluster resource type can’t be altered after.
158
156
159
157
For Application Insights CMK configuration, follow the Appendix content.
160
158
161
-
You must specify the capacity reservation level (sku) for the *Cluster* resource when creating a *Cluster* resource. The capacity reservation level can be in the range of 1000 to 2000 and you can update it in steps of 100 later. If you need capacity reservation level higher than 2000, reach your Microsoft contact to enable it. This property doesn’t affect billing currently -- once pricing model for dedicated cluster is introduced, billing will apply to any existing CMK deployments.
159
+
You must specify the capacity reservation level (sku) when creating a *Cluster* resource. The capacity reservation level can be in the range of 1,000 to 2,000 GB per day and you can update it in steps of 100 later. If you need capacity reservation level higher than 2,000 GB per day, reach your Microsoft contact to enable it. This property doesn’t affect billing currently -- once pricing model for dedicated cluster is introduced, billing will apply to any existing CMK deployments.
162
160
163
161
**Create**
164
162
@@ -188,7 +186,7 @@ The identity is assigned to the *Cluster* resource at creation time.
188
186
202 Accepted. This is a standard Resource Manager response for asynchronous operations.
189
187
190
188
>[!Important]
191
-
> It takes the provisioning of the underly ADX cluster a few minutes to complete. You can verify the provisioning state when performing GET REST API call on the *Cluster* resource and looking at the *provisioningState* value. It is *ProvisioningAccount* while provisioning and "Succeeded" when completed.
189
+
> It takes the provisioning of the underly ADX cluster a while to complete. You can verify the provisioning state when performing GET REST API call on the *Cluster* resource and looking at the *provisioningState* value. It is *ProvisioningAccount* while provisioning and *Succeeded* when completed.
> Copy and save the response since you will need these details in later steps
201
+
> Copy and save the response since you will need its details in later steps
204
202
205
203
**Response**
206
204
```json
@@ -242,7 +240,7 @@ The *Get* permission is required to verify that your Key Vault is configured as
242
240
243
241
### Update Cluster resource with Key identifier details
244
242
245
-
This step applies per initial and future key version updates in your Key Vault. It informs Azure Monitor Storage about the new key version.
243
+
This step applies per initial and future key version updates in your Key Vault. It informs Azure Monitor Storage about the key version to be used for data encryption. When updated, your new key is being used to wrap and unwrap to Storage key (AEK).
246
244
247
245
To update the *Cluster* resource with your Key Vault *Key identifier* details, select the current version of your key in Azure Key Vault to get the Key identifier details.
After the workspaces association, data ingested to your workspaces is stored encrypted with your managed key.
352
+
The workspace association is performed via Resource Manager asynchronous operations, which can take up to 90 minutes to complete. The next step shows you how workspace association state can be checked. After the workspaces association, data ingested to your workspaces is stored encrypted with your managed key.
355
353
356
354
### Workspace association verification
357
-
You can verify if a workspace is associated to a *Custer* resource by looking at the [Workspaces – Get](https://docs.microsoft.com/rest/api/loganalytics/workspaces/get) response. Associated workspace will have a 'clusterResourceId' property with the*Cluster* resource ID.
355
+
You can verify if a workspace is associated to a *Cluster* resource by looking at the [Workspaces – Get](https://docs.microsoft.com/rest/api/loganalytics/workspaces/get) response. Associated workspaces will have a 'clusterResourceId' property with a*Cluster* resource ID.
358
356
359
357
```rest
360
358
GET https://management.azure.com/subscriptions/<subscription-id>/resourcegroups/<resource-group-name>/providers/microsoft.operationalInsights/workspaces/<workspace-name>?api-version=2015-11-01-preview
@@ -394,7 +392,7 @@ GET https://management.azure.com/subscriptions/<subscription-id>/resourcegroups/
394
392
395
393
## CMK (KEK) revocation
396
394
397
-
You can revoke your access to your data by disabling your key or deleting the *Cluster* resource access policy in your Key Vault. Azure Monitor Storage will always respect changes in key permissions within an hour, normally sooner, and Storage will become unavailable. Any data ingested to workspaces associated with your *Cluster* resource is dropped and queries will fail. Previously ingested data remains inaccessible in Azure Monitor Storage as long as your key is revoked, and your workspaces aren't deleted. Inaccessible data is governed by the data-retention policy and will be purged when retention is reached.
395
+
You can revoke your access to your data by disabling your key or deleting the *Cluster* resource access policy in your Key Vault. Azure Monitor Storage will always respect changes in key permissions within an hour, normally sooner, and Storage will become unavailable. Any data ingested to workspaces associated with your *Cluster* resource is dropped and queries will fail. Previously ingested data remains inaccessible in Azure Monitor Storage as long as your your *Cluster* resource and your workspaces aren't deleted. Inaccessible data is governed by the data-retention policy and will be purged when retention is reached.
398
396
399
397
Storage will periodically poll your Key Vault to attempt to unwrap the
400
398
encryption key and once accessed, data ingestion and query resume within
@@ -403,24 +401,21 @@ encryption key and once accessed, data ingestion and query resume within
403
401
## CMK (KEK) rotation
404
402
405
403
Rotation of CMK requires explicit update of the *Cluster* resource with the new key version in Azure Key Vault. To update Azure Monitor with your new key version, follow the instructions in "Update *Cluster* resource with Key identifier details" step. If you update your key version in Key Vault and don't update the new Key identifier details in the *Cluster* resource, Azure Monitor Storage will keep using your previous key.
406
-
All your data is accessible after the key rotation operation including data ingested before the rotation and after it, since all data remains encrypted by the Account Encryption Key (AEK) while it’s now being encrypted by your new Key Encryption Key (KEK) version.
404
+
All your data is accessible after the key rotation operation including data ingested before the rotation and after it, since all data remains encrypted by the Account Encryption Key (AEK) while AEK is now being encrypted by your new Key Encryption Key (KEK) version.
407
405
408
406
## Limitations and constraints
409
407
410
408
- The CMK feature is supported at ADX cluster level and requires a
411
-
dedicated Azure Monitor ADX cluster
409
+
dedicated Azure Monitor ADX cluster with requirement of sending 1TB per day or more.
412
410
413
411
- The max number of *Cluster* resources per subscription is limited to 2
414
412
415
-
-*Cluster* resource association to workspace should be carried ONLY
416
-
after you received a confirmation from the product group that the
417
-
ADX cluster provisioning was fulfilled. Data that is sent prior to
418
-
this provisioning will be dropped and won't be recoverable.
413
+
-*Cluster* resource association to workspace should be carried ONLY after you have verified that the ADX cluster provisioning was fulfilled. Data that is sent prior to this provisioning will be dropped and won't be recoverable.
419
414
420
415
- CMK encryption applies to newly ingested data after the CMK
421
416
configuration. Data that was ingested prior to the CMK
422
417
configuration, remains encrypted with Microsoft key. You can query
423
-
data before and after the CMK configuration seamlessly.
418
+
data ingested before and after the CMK configuration seamlessly.
424
419
425
420
- Once workspace is associated to a *Cluster* resource, it cannot be
426
421
de-associated from the *Cluster* resource, since data is encrypted
@@ -447,11 +442,11 @@ All your data is accessible after the key rotation operation including data inge
447
442
## Troubleshooting and management
448
443
449
444
- Key Vault availability
450
-
- In normal operation, Storage caches AEK for short periods of time periodically goes back to Key Vault to unwrap.
445
+
- In normal operation -- Storage caches AEK for short periods of time and goes back to Key Vault to unwrap periodically.
451
446
452
-
- Transient connection errors. Storage handles transient errors (timeouts, connection failures, DNS issues) by allowing keys to stay in cache for a short while longer and this overcomes any small blips in availability. The query and ingestion capabilities continue without interruption.
447
+
- Transient connection errors -- Storage handles transient errors (timeouts, connection failures, DNS issues) by allowing keys to stay in cache for a short while longer and this overcomes any small blips in availability. The query and ingestion capabilities continue without interruption.
453
448
454
-
- Live site, unavailability of about 30 minutes will cause the Storage account to become unavailable. The query capability is unavailable and ingested data is cached for several hours using Microsoft key to avoid data loss. When access to Key Vault is restored, query becomes available and the temporary cached data is ingested to the data-store and encrypted with CMK.
449
+
- Live site -- unavailability of about 30 minutes will cause the Storage account to become unavailable. The query capability is unavailable and ingested data is cached for several hours using Microsoft key to avoid data loss. When access to Key Vault is restored, query becomes available and the temporary cached data is ingested to the data-store and encrypted with CMK.
455
450
456
451
- If you create a *Cluster* resource and specify the KeyVaultProperties immediately, the operation may fail since the
457
452
access policy can't be defined until system identity is assigned to the *Cluster* resource.
@@ -508,7 +503,8 @@ All your data is accessible after the key rotation operation including data inge
508
503
509
504
The same response as for '*Cluster* resources for a resource group', but in subscription scope.
510
505
511
-
- Delete a *Cluster* resource -- a soft-delete operation is performed to allow the recovery of your *Cluster* resource, your data and associated workspaces within 14 days, whether the deletion was accidental or intentional. After the soft-delete period, your *Cluster* resource and data are non-recoverable. The *Cluster* resource name remains reserved during the soft-delete period and you can’t create a new cluster with that name.
506
+
- Delete your *Cluster* resource -- a soft-delete operation is performed to allow the recovery of your Cluster resource, your data and associated workspaces within 14 days, whether the deletion was accidental or intentional. The *Cluster* resource name remains reserved during the soft-delete period and you can’t create a new cluster with that name.
507
+
After the soft-delete period, your *Cluster* resource and data are non-recoverable. Associated workspaces are de-associated from the *Cluster* resource and new data is ingested to shared Storage and encrypted with Microsoft key.
0 commit comments