|
2 | 2 | title: Overview of Azure Service Bus throttling | Microsoft Docs
|
3 | 3 | description: Overview of Service Bus throttling - Standard and Premium tiers.
|
4 | 4 | ms.topic: article
|
5 |
| -ms.date: 09/20/2021 |
| 5 | +ms.date: 11/14/2022 |
6 | 6 | ---
|
7 | 7 |
|
8 | 8 | # Throttling operations on Azure Service Bus
|
9 | 9 |
|
10 |
| -Cloud native solutions give a notion of unlimited resources that can scale with your workload. While this notion is more true in the cloud than it is with on-premises systems, there are still limitations that exist in the cloud. |
| 10 | +Cloud native solutions give a notion of unlimited resources that can scale with your workload. While this notion is more true in the cloud than it is with on-premises systems, there are still limitations that exist in the cloud. These limitations may cause throttling of client application requests in both standard and premium tiers as discussed in this article. |
11 | 11 |
|
12 |
| -These limitations may cause throttling of client application requests in both Standard and Premium tiers as discussed in this article. |
| 12 | +## Throttling in standard tier |
13 | 13 |
|
14 |
| -## Throttling in Azure Service Bus Standard tier |
| 14 | +The standard tier of Service Bus operates as a multi-tenant setup with a pay-as-you-go pricing model. Here multiple namespaces in the same cluster share the allocated resources. Standard tier is the recommended choice for developer environments, QA environments, and low throughput production systems. |
15 | 15 |
|
16 |
| -The Azure Service Bus Standard tier operates as a multi-tenant setup with a pay-as-you-go pricing model. Here multiple namespaces in the same cluster share the allocated resources. Standard tier is the recommended choice for developer, testing, and QA environments along with low throughput production systems. |
| 16 | +In the past, Service Bus had coarse throttling limits strictly dependent on resource utilization. However, there's an opportunity to refine throttling logic and provide predictable throttling behavior to all namespaces that are sharing these resources. |
17 | 17 |
|
18 |
| -In the past, Azure Service Bus had coarse throttling limits strictly dependent on resource utilization. However, there is an opportunity to refine the throttling logic and provide predictable throttling behavior to all namespaces that are sharing these resources. |
19 |
| - |
20 |
| -In an attempt to ensure fair usage and distribution of resources across all the Azure Service Bus Standard namespaces that use the same resources, the throttling logic has been modified to be credit-based. |
| 18 | +In an attempt to ensure fair usage and distribution of resources across all the Service Bus standard namespaces that use the same resources, the throttling logic has been modified to be credit-based. |
21 | 19 |
|
22 | 20 | > [!NOTE]
|
23 |
| -> It is important to note that throttling is ***not new*** to Azure Service Bus, or any cloud native service. |
| 21 | +> It is important to note that throttling is **not new** to Azure Service Bus, or any cloud native service. |
24 | 22 | >
|
25 |
| -> Credit based throttling is simply refining the way various namespaces share resources in a multi-tenant Standard tier environment and thus enabling fair usage by all namespaces sharing the resources. |
| 23 | +> Credit based throttling is simply refining the way various namespaces share resources in a multi-tenant standard tier environment and thus enabling fair usage by all namespaces sharing the resources. |
26 | 24 |
|
27 | 25 | ### What is credit-based throttling?
|
28 | 26 |
|
29 |
| -Credit-based throttling limits the number of operations that can be performed on a given namespace in a specific time period. |
30 |
| - |
31 |
| -Below is the workflow for credit-based throttling - |
| 27 | +Credit-based throttling limits the number of operations that can be performed on a given namespace in a specific time period. Here's the workflow for credit-based throttling. |
32 | 28 |
|
33 |
| - * At the start of each time period, we provide a certain number of credits to each namespace. |
| 29 | + * At the start of each time period, Service Bus provides a certain number of credits to each namespace. |
34 | 30 | * Any operations performed by the sender or receiver client applications will be counted against these credits (and subtracted from the available credits).
|
35 | 31 | * If the credits are depleted, subsequent operations will be throttled until the start of the next time period.
|
36 | 32 | * Credits are replenished at the start of the next time period.
|
37 | 33 |
|
38 | 34 | ### What are the credit limits?
|
39 | 35 |
|
40 |
| -The credit limits are currently set to '1000' credits every second (per namespace). |
41 |
| - |
42 |
| -Not all operations are created equal. Here are the credit costs of each of the operations - |
| 36 | +The credit limits are currently set to **1000 credits every second** (per namespace). Not all operations are created equal. Here are the credit costs of each of the operations |
43 | 37 |
|
44 | 38 | | Operation | Credit cost|
|
45 | 39 | |-----------|-----------|
|
46 |
| -| Data operations (Send, SendAsync, Receive, ReceiveAsync, Peek) |1 credit per message | |
47 |
| -| Management operations (Create, Read, Update, Delete on Queues, Topics, Subscriptions, Filters) | 10 credits | |
| 40 | +| Data operations (`Send`, `SendAsync`, `Receive`, `ReceiveAsync`, `Peek`) | 1 credit per message | |
| 41 | +| Management operations (`Create`, `Read`, `Update`, `Delete` on queues, topics, subscriptions, filters) | 10 credits | |
48 | 42 |
|
49 | 43 | > [!NOTE]
|
50 |
| -> Please note that when sending to a Topic, each message is evaluated against filter(s) before being made available on the Subscription. |
51 |
| -> Each filter evaluation also counts against the credit limit (i.e. 1 credit per filter evaluation). |
52 |
| -> |
| 44 | +> When sending to a topic, each message is evaluated against filters before being made available on the subscription. Each filter evaluation also counts against the credit limit (that is, 1 credit per filter evaluation). |
53 | 45 |
|
54 | 46 | ### How will I know that I'm being throttled?
|
55 | 47 |
|
56 |
| -When the client application requests are being throttled, the below server response will be received by your application and logged. |
| 48 | +When the client application requests are being throttled, the client application receives the following server response. |
57 | 49 |
|
58 | 50 | ```
|
59 | 51 | The request was terminated because the entity is being throttled. Error code: 50009. Please wait 2 seconds and try again.
|
60 | 52 | ```
|
61 | 53 |
|
62 | 54 | ### How can I avoid being throttled?
|
63 | 55 |
|
64 |
| -With shared resources, it is important to enforce some sort of fair usage across various Service Bus namespaces that share those resources. Throttling ensures that any spike in a single workload does not cause other workloads on the same resources to be throttled. |
65 |
| - |
66 |
| -As mentioned later in the article, there is no risk in being throttled because the client SDKs (and other Azure PaaS offerings) have the default retry policy built into them. Any throttled requests will be retried with exponential backoff and will eventually go through when the credits are replenished. |
| 56 | +With shared resources, it's important to enforce some sort of fair usage across various Service Bus namespaces that share those resources. Throttling ensures that any spike in a single workload doesn't cause other workloads on the same resources to be throttled. As mentioned later in the article, there's no risk in being throttled because the client SDKs (and other Azure PaaS offerings) have the default retry policy built into them. Any throttled requests will be retried with exponential backoff and will eventually go through when the credits are replenished. |
67 | 57 |
|
68 |
| -Understandably, some applications may be sensitive to being throttled. In that case, it is recommended to [migrate your current Service Bus Standard namespace to Premium](service-bus-migrate-standard-premium.md). |
| 58 | +Understandably, some applications may be sensitive to being throttled. In that case, it's recommended to [migrate your current Service Bus standard namespace to premium](service-bus-migrate-standard-premium.md). On migration, you can allocate dedicated resources to your Service Bus namespace and appropriately scale up the resources if there's a spike in your workload and reduce the likelihood of being throttled. Additionally, when your workload reduces to normal levels, you can scale down the resources allocated to your namespace. |
69 | 59 |
|
70 |
| -On migration, you can allocate dedicated resources to your Service Bus namespace and appropriately scale up the resources if there is a spike in your workload and reduce the likelihood of being throttled. Additionally, when your workload reduces to normal levels, you can scale down the resources allocated to your namespace. |
| 60 | +## Throttling in premium tier |
71 | 61 |
|
72 |
| -## Throttling in Azure Service Bus Premium tier |
73 |
| - |
74 |
| -The [Azure Service Bus Premium](service-bus-premium-messaging.md) tier allocates dedicated resources, in terms of messaging units, to each namespace setup by the customer. These dedicated resources provide predictable throughput and latency and are recommended for high throughput or sensitive production grade systems. |
75 |
| - |
76 |
| -Additionally, the Premium tier also enables customers to scale up their throughput capacity when they experience spikes in the workload. |
| 62 | +The [Service Bus premium](service-bus-premium-messaging.md) tier allocates dedicated resources, in terms of messaging units, to each namespace setup by the customer. These dedicated resources provide predictable throughput and latency and are recommended for high throughput or sensitive production grade systems. Additionally, the premium tier also enables customers to scale up their throughput capacity when they experience spikes in the workload. For more information, see [Automatically update messaging units of an Azure Service Bus namespace](automate-update-messaging-units.md). |
77 | 63 |
|
78 | 64 | ### How does throttling work in Service Bus Premium?
|
79 | 65 |
|
80 |
| -With exclusive resource allocation for Service Bus Premium, throttling is purely driven by the limitations of the resources allocated to the namespace. |
81 |
| - |
82 |
| -If the number of requests are more than the current resources can service, then the requests will be throttled. |
| 66 | +With exclusive resource allocation for the premium tier, throttling is purely driven by the limitations of the resources allocated to the namespace. If the number of requests are more than the current resources can service, then the requests will be throttled. |
83 | 67 |
|
84 | 68 | ### How will I know that I'm being throttled?
|
85 | 69 |
|
86 |
| -There are various ways to identifying throttling in Azure Service Bus Premium - |
| 70 | +There are various ways to identifying throttling in the Service Bus premium tier. |
| 71 | + |
87 | 72 | * **Throttled Requests** show up on the [Azure Monitor Request metrics](monitor-service-bus-reference.md#request-metrics) to identify how many requests were throttled.
|
88 | 73 | * High **CPU Usage** indicates that current resource allocation is high and requests may get throttled if the current workload doesn't reduce.
|
89 | 74 | * High **Memory Usage** indicates that current resource allocation is high and requests may get throttled if the current workload doesn't reduce.
|
90 | 75 |
|
91 | 76 | ### How can I avoid being throttled?
|
92 | 77 |
|
93 |
| -Since the Service Bus Premium namespace already has dedicated resources, you can reduce the possibility of getting throttled by scaling up the number of Messaging Units allocated to your namespace in the event (or anticipation) of a spike in the workload. |
| 78 | +As the Service Bus premium namespace already has dedicated resources, you can reduce the possibility of getting throttled by scaling up the number of messaging units allocated to your namespace in the event (or anticipation) of a spike in the workload. For more information, see [Automatically update messaging units of an Azure Service Bus namespace](automate-update-messaging-units.md). |
94 | 79 |
|
95 |
| -Scaling up/down can be achieved by creating [runbooks](../automation/automation-create-alert-triggered-runbook.md) that can be triggered by changes in the above metrics. |
96 | 80 |
|
97 | 81 | ## FAQs
|
98 | 82 |
|
99 | 83 | ### How does throttling affect my application?
|
100 | 84 |
|
101 |
| -When a request is throttled, it implies that the service is busy because it is facing more requests than the resources allow. If the same operation is tried again after a few moments, once the service has worked through its current workload, then the request can be honored. |
102 |
| - |
103 |
| -Since throttling is the expected behavior of any cloud native service, we have built the retry logic into the Azure Service Bus SDK itself. The default is set to auto retry with an exponential back-off to ensure that we don't have the same request being throttled each time. |
| 85 | +When a request is throttled, it implies that the service is busy because it's facing more requests than the resources allow. If the same operation is tried again after a few moments, once the service has worked through its current workload, then the request can be honored. |
104 | 86 |
|
105 |
| -The default retry logic will apply to every operation. |
| 87 | +As throttling is the expected behavior of any cloud native service, retry logic is built into the Service Bus SDK itself. The default is set to auto retry with an exponential back-off to ensure that we don't have the same request being throttled each time. The default retry logic will apply to every operation. |
106 | 88 |
|
107 | 89 | ### Does throttling result in data loss?
|
108 | 90 |
|
109 | 91 | Azure Service Bus is optimized for persistence, we ensure that all the data sent to Service Bus is committed to storage before the service acknowledges the success of the request.
|
110 | 92 |
|
111 |
| -Once the request is successfully 'ACK' (acknowledged) by Service Bus, it implies that Service Bus has successfully processed the request. If Service Bus returns a 'NACK' (failure), then it implies that Service Bus has not been able to process the request and the client application must retry the request. |
| 93 | +Once the request is successfully acknowledged by Service Bus, it implies that Service Bus has successfully processed the request. If Service Bus returns a failure, then it implies that Service Bus hasn't been able to process the request and the client application must retry the request. |
112 | 94 |
|
113 |
| -However, when a request is throttled, the service is implying that it cannot accept and process the request right now because of resource limitations. This **does not** imply any sort of data loss because Service Bus simply hasn't looked at the request. In this case, relying on the default retry policy of the Service Bus SDK ensures that the request is eventually processed. |
| 95 | +However, when a request is throttled, the service is implying that it can't accept and process the request right now because of resource limitations. It **does not** imply any sort of data loss because Service Bus simply hasn't looked at the request. In this case, relying on the default retry policy of the Service Bus SDK ensures that the request is eventually processed. |
114 | 96 |
|
115 | 97 | ## Next steps
|
116 | 98 |
|
|
0 commit comments