You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/key-vault/key-vault-ovw-throttling.md
+40-90Lines changed: 40 additions & 90 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,11 +4,10 @@ description: Key Vault throttling limits the number of concurrent calls to preve
4
4
services: key-vault
5
5
author: msmbaldwin
6
6
manager: rkarlin
7
-
tags:
8
7
9
8
ms.service: key-vault
10
9
ms.topic: conceptual
11
-
ms.date: 05/10/2018
10
+
ms.date: 12/02/2019
12
11
ms.author: mbaldwin
13
12
14
13
---
@@ -21,10 +20,34 @@ Throttling limits vary based on the scenario. For example, if you are performing
21
20
22
21
## How does Key Vault handle its limits?
23
22
24
-
Service limits in Key Vault are there to prevent misuse of resources and ensure quality of service for all of Key Vault’s clients. When a service threshold is exceeded, Key Vault limits any further requests from that client for a period of time. When this happens, Key Vault returns HTTP status code 429 (Too many requests), and the requests fail. Also, failed requests that return a 429 count towards the throttle limits tracked by Key Vault.
23
+
Service limits in Key Vault prevent misuse of resources and ensure quality of service for all of Key Vault's clients. When a service threshold is exceeded, Key Vault limits any further requests from that client for a period of time, returns HTTP status code 429 (Too many requests), and the request fails. Failed requests that return a 429 count towards the throttle limits tracked by Key Vault.
25
24
26
-
If you have a valid business case for higher throttle limits, please contact us.
25
+
Key Vault was originally designed to be used to store and retrieve your secrets at deployment time. The world has evolved, and Key Vault is being used at run-time to store and retrieve secrets, and often apps and services want to use Key Vault like a database. Current limits do not support high throughput rates.
26
+
27
+
Key Vault was originally created with the limits specified in [Azure Key Vault service limits](key-vault-service-limits.md). To maximize your Key Vault through put rates, here are some recommended guidelines/best practices for maximizing your throughput:
28
+
1. Ensure you have throttling in place. Client must honor exponential back-off policies for 429's and ensure you are doing retries as per the guidance below.
29
+
1. Divide your Key Vault traffic amongst multiple vaults and different regions. Use a separate vault for each security/availability domain. If you have five apps, each in two regions, then we recommend 10 vaults each containing the secrets unique to app and region. A subscription-wide limit for all transaction types is five times the individual key vault limit. For example, HSM-other transactions per subscription are limited to 5,000 transactions in 10 seconds per subscription. Consider caching the secret within your service or app to also reduce the RPS directly to key vault and/or handle burst based traffic. You can also divide your traffic amongst different regions to minimize latency and use a different subscription/vault. Do not send more than the subscription limit to the Key Vault service in a single Azure region.
30
+
1. Cache the secrets you retrieve from Azure Key Vault in memory, and reuse from memory whenever possible. Re-read from Azure Key Vault only when the cached copy stops working (e.g. because it got rotated at the source).
31
+
1. Key Vault is designed for your own services secrets. If you are storing your customers' secrets (especially for high-throughput key storage scenarios), consider putting the keys in a database or storage account with encryption, and storing just the master key in Azure Key Vault.
32
+
1. Encrypt, wrap, and verify public-key operations can be performed with no access to Key Vault, which not only reduces risk of throttling, but also improves reliability (as long as you properly cache the public key material).
33
+
1. If you use Key Vault to store credentials for a service, check if that service supports AAD Authentication to authenticate directly. This reduces the load on Key Vault, improves reliability and simplifies your code since Key Vault can now use the AAD token. Many services have moved to using AAD Auth. See the current list at [Services that support managed identities for Azure resources](../active-directory/managed-identities-azure-resources/services-support-managed-identities.md#azure-services-that-support-managed-identities-for-azure-resources).
34
+
1. Consider staggering your load/deployment over a longer period of time to stay under the current RPS limits.
35
+
1. If your app comprises multiple nodes that need to read the same secret(s), then consider using a fan out pattern, where one entity reads the secret from Key Vault, and fans out to all nodes. Cache the retrieved secrets only in memory.
36
+
If you find that the above still does not meet your needs, please fill out the below table and contact us to determine what additional capacity can be added (example put below for illustrative purposes only).
37
+
38
+
| Vault name | Vault Region | Object type (Secret, Key, or Cert) | Operation(s)*| Key Type | Key Length or Curve | HSM key?| Steady state RPS needed | Peak RPS needed |
\* For a full list of possible values, see [Azure Key Vault operations](/rest/api/keyvault/key-operations).
43
+
44
+
If additional capacity is approved, please note the following as result of the capacity increases:
45
+
1. Data consistency model changes. Once a vault is allow listed with additional throughput capacity, the Key Vault service data consistency guarantee changes (necessary to meet higher volume RPS since the underlying Azure Storage service cannot keep up). In a nutshell:
46
+
1.**Without allow listing**: The Key Vault service will reflect the results of a write operation (eg. SecretSet, CreateKey) immediately in subsequent calls (eg. SecretGet, KeySign).
47
+
1.**With allow listing**: The Key Vault service will reflect the results of a write operation (eg. SecretSet, CreateKey) within 60 seconds in subsequent calls (eg. SecretGet, KeySign).
48
+
1. Client code must honor back-off policy for 429 retries. The client code calling the Key Vault service must not immediately retry Key Vault requests when it receives a 429 response code. The Azure Key Vault throttling guidance published here recommends applying exponential backoff when receiving a 429 Http response code.
49
+
50
+
If you have a valid business case for higher throttle limits, please contact us.
28
51
29
52
## How to throttle your app in response to service limits
30
53
@@ -38,97 +61,24 @@ When you implement your app's error handling, use the HTTP error code 429 to det
38
61
39
62
Code that implements exponential backoff is shown below.
40
63
```
41
-
public sealed class RetryWithExponentialBackoff
64
+
SecretClientOptions options = new SecretClientOptions()
42
65
{
43
-
private readonly int maxRetries, delayMilliseconds, maxDelayMilliseconds;
44
-
45
-
public RetryWithExponentialBackoff(int maxRetries = 50,
46
-
int delayMilliseconds = 200,
47
-
int maxDelayMilliseconds = 2000)
48
-
{
49
-
this.maxRetries = maxRetries;
50
-
this.delayMilliseconds = delayMilliseconds;
51
-
this.maxDelayMilliseconds = maxDelayMilliseconds;
52
-
}
53
-
54
-
public async Task RunAsync(Func<Task> func)
66
+
Retry =
55
67
{
56
-
ExponentialBackoff backoff = new ExponentialBackoff(this.maxRetries,
57
-
this.delayMilliseconds,
58
-
this.maxDelayMilliseconds);
59
-
retry:
60
-
try
61
-
{
62
-
await func();
63
-
}
64
-
catch (Exception ex) when (ex is TimeoutException ||
65
-
ex is System.Net.Http.HttpRequestException)
66
-
{
67
-
Debug.WriteLine("Exception raised is: " +
68
-
ex.GetType().ToString() +
69
-
" –Message: " + ex.Message +
70
-
" -- Inner Message: " +
71
-
ex.InnerException.Message);
72
-
await backoff.Delay();
73
-
goto retry;
74
-
}
75
-
}
76
-
}
77
-
78
-
public struct ExponentialBackoff
79
-
{
80
-
private readonly int m_maxRetries, m_delayMilliseconds, m_maxDelayMilliseconds;
81
-
private int m_retries, m_pow;
82
-
83
-
public ExponentialBackoff(int maxRetries, int delayMilliseconds,
84
-
int maxDelayMilliseconds)
85
-
{
86
-
m_maxRetries = maxRetries;
87
-
m_delayMilliseconds = delayMilliseconds;
88
-
m_maxDelayMilliseconds = maxDelayMilliseconds;
89
-
m_retries = 0;
90
-
m_pow = 1;
91
-
}
92
-
93
-
public Task Delay()
94
-
{
95
-
if (m_retries == m_maxRetries)
96
-
{
97
-
throw new TimeoutException("Max retry attempts exceeded.");
If you need more help at any point in this article, you can contact the Azure experts on the [MSDN Azure and Stack Overflow forums](https://azure.microsoft.com/support/forums/). Alternatively, you can file an Azure support incident. Go to the [Azure support site](https://azure.microsoft.com/support/options/) and select Get support. For information about using Azure Support, read the [Microsoft Azure support FAQ](https://azure.microsoft.com/support/faq/).
208
+
If you need more help at any point in this article, you can contact the Azure experts on the [MSDN Azure and Stack Overflow forums](https://azure.microsoft.com/support/forums/). Alternatively, you can file an Azure support incident. Go to the [Azure support site](https://azure.microsoft.com/support/options/) and select Get support. For information about using Azure Support, read the [Microsoft Azure support FAQ](https://azure.microsoft.com/support/faq/).
0 commit comments