Skip to content

Commit 74efefc

Browse files
committed
Finish the scaling article
1 parent f34c8c4 commit 74efefc

File tree

1 file changed

+103
-28
lines changed

1 file changed

+103
-28
lines changed

articles/azure-functions/functions-target-based-scaling.md

Lines changed: 103 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -4,18 +4,18 @@ description: Explains target-based scaling behaviors of Consumption plan and Pre
44
ms.date: 04/04/2023
55
ms.topic: conceptual
66
ms.service: azure-functions
7-
87
---
8+
99
# Target-based scaling
1010

1111
Target-based scaling provides a fast and intuitive scaling model for customers and is currently supported for the following extensions:
1212

13-
- Service Bus Queues and Topics
13+
- Service Bus queues and topics
1414
- Storage Queues
1515
- Event Hubs
16-
- Cosmos DB
16+
- Azure Cosmos DB
1717

18-
Target-based scaling replaces the previous Azure Functions incremental scaling model as the default for these extension types. Incremental scaling added or removed a maximum of 1 worker at [each new instance rate](event-driven-scaling.md#understanding-scaling-behaviors), with complex decisions for when to scale. In contrast, target-based scaling allows scale up of 4 instances at a time, and the scaling decision is based on a simple target-based equation:
18+
Target-based scaling replaces the previous Azure Functions incremental scaling model as the default for these extension types. Incremental scaling added or removed a maximum of one worker at [each new instance rate](event-driven-scaling.md#understanding-scaling-behaviors), with complex decisions for when to scale. In contrast, target-based scaling allows scale up of four instances at a time, and the scaling decision is based on a simple target-based equation:
1919

2020
![Desired instances = event source length / target executions per instance](./media/functions-target-based-scaling/targetbasedscalingformula.png)
2121

@@ -29,7 +29,8 @@ The default _target executions per instance_ values come from the SDKs used by t
2929
Target-based scaling is supported for the [Consumption](consumption-plan.md) and [Premium](functions-premium-plan.md) plans. Your function app runtime must be 4.3.0 or higher.
3030

3131
## Opting out
32-
Target-based scaling is on by default for apps on the Consumption plan or Premium plans without runtime scale monitoring. If you wish to disable target-based scaling and revert to incremental scaling, add the following app setting to your function app:
32+
33+
Target-based scaling is enabled by default for function apps on the Consumption plan or Premium plans without runtime scale monitoring. If you wish to disable target-based scaling and revert to incremental scaling, add the following app setting to your function app:
3334

3435
| App Setting | Value |
3536
| ----------------------------- | ----- |
@@ -60,17 +61,18 @@ For Azure Cosmos DB _target executions per instance_ is set in the function attr
6061
| ----------------- | ------------------------ | ------------- |
6162
| Azure Cosmos DB | maxItemsPerInvocation | 100 |
6263

63-
See [Details per extension](#details-per-extension) for example configurations of the supported extensions.
64+
To learn more, see the [example configurations for the supported extensions](#details-per-extension).
6465

6566
## Premium plan with runtime scale monitoring enabled
67+
6668
In [runtime scale monitoring](functions-networking-options.md?tabs=azure-cli#premium-plan-with-virtual-network-triggers), the extensions handle target-based scaling. Hence, in addition to the function app runtime version requirement, your extension packages must meet the following minimum versions:
6769

6870
| Extension Name | Minimum Version Needed |
6971
| -------------- | ---------------------- |
7072
| Storage Queue | 5.1.0 |
7173
| Event Hubs | 5.2.0 |
7274
| Service Bus | 5.9.0 |
73-
| Cosmos DB | 4.1.0 |
75+
| Azure Cosmos DB | 4.1.0 |
7476

7577
Additionally, target-based scaling is currently an **opt-in** feature with runtime scale monitoring. In order to use target-based scaling with the Premium plan when runtime scale monitoring is enabled, add the following app setting to your function app:
7678

@@ -79,9 +81,13 @@ Additionally, target-based scaling is currently an **opt-in** feature with runti
7981
|`TARGET_BASED_SCALING_ENABLED` | 1 |
8082

8183
## Dynamic concurrency support
82-
Target-based scaling introduces faster scaling, and uses defaults for _target executions per instance_. When using Service Bus or Storage queues you also have the option of enabling [dynamic concurrency](functions-concurrency.md#dynamic-concurrency). In this configuration, the _target executions per instance_ value is determined automatically by the dynamic concurrency feature. It starts with limited concurrency and identifies the best setting over time.
8384

84-
## Details per extension
85+
Target-based scaling introduces faster scaling, and uses defaults for _target executions per instance_. When using Service Bus or Storage queues, you can also enable [dynamic concurrency](functions-concurrency.md#dynamic-concurrency). In this configuration, the _target executions per instance_ value is determined automatically by the dynamic concurrency feature. It starts with limited concurrency and identifies the best setting over time.
86+
87+
## Supported extensions
88+
89+
The way in which you configure target-based scaling in your host.json file depends on the specific extension type. This section provides the configuration details for the extensions that currently support target-based scaling.
90+
8591
### Service Bus queues and topics
8692

8793
The Service Bus extension support three execution models, determined by the `IsBatched` and `IsSessionsEnabled` attributes of your Service Bus trigger. The default value for `IsBatched` and `IsSessionsEnabled` is `false`.
@@ -93,13 +99,17 @@ The Service Bus extension support three execution models, determined by the `IsB
9399
| Batch Processing | true | false | maxMessageBatchSize or maxMessageCount |
94100

95101
> [!NOTE]
96-
> **Scale efficiency:** For the Service Bus extension, use _Manage_ rights on resources for the most efficient scaling. With _Listen_ rights scaling will revert to incremental scale because the queue or topic length can't be used to inform scaling decisions. To learn more about setting rights in Service Bus access policies, see [Shared Access Authorization Policy](../service-bus-messaging/service-bus-sas.md#shared-access-authorization-policies).
102+
> **Scale efficiency:** For the Service Bus extension, use _Manage_ rights on resources for the most efficient scaling. With _Listen_ rights scaling reverts to incremental scale because the queue or topic length can't be used to inform scaling decisions. To learn more about setting rights in Service Bus access policies, see [Shared Access Authorization Policy](../service-bus-messaging/service-bus-sas.md#shared-access-authorization-policies).
97103
98104

99105
#### Single Dispatch Processing
100-
In this model, each invocation of your function processes a single message. The `maxConcurrentCalls` setting governs _target executions per instance_.
101106

102-
For **v5.x+** of the Service Bus extension, modify the `host.json` setting `maxConcurrentCalls`:
107+
In this model, each invocation of your function processes a single message. The `maxConcurrentCalls` setting governs _target executions per instance_. The specific setting depends on the version of the Service Bus extension.
108+
109+
# [v5.x+](#tab/v5)
110+
111+
Modify the `host.json` setting `maxConcurrentCalls`, as in the following example:
112+
103113
```json
104114
{
105115
"version": "2.0",
@@ -111,7 +121,10 @@ For **v5.x+** of the Service Bus extension, modify the `host.json` setting `maxC
111121
}
112122
```
113123

114-
For Functions host **v2.x+**, modify the `host.json` setting `maxConcurrentCalls` in `messageHandlerOptions`:
124+
# [v2.x+](#tab/v2)
125+
126+
Modify the `host.json` setting `maxConcurrentCalls` in `messageHandlerOptions`, as in the following example:
127+
115128
```json
116129
{
117130
"version": "2.0",
@@ -124,10 +137,16 @@ For Functions host **v2.x+**, modify the `host.json` setting `maxConcurrentCalls
124137
}
125138
}
126139
```
140+
---
141+
127142
#### Single Dispatch Processing (Session Based)
128-
In this model, each invocation of your function processes a single message. However, depending on the number of active sessions for your Service Bus topic or queue, each instance leases one or more sessions.
129143

130-
For **v5.x+** of the Service Bus extension, modify the `host.json` setting `maxConcurrentSessions` to set _target executions per instance_:
144+
In this model, each invocation of your function processes a single message. However, depending on the number of active sessions for your Service Bus topic or queue, each instance leases one or more sessions. The specific setting depends on the version of the Service Bus extension.
145+
146+
# [v5.x+](#tab/v5)
147+
148+
Modify the `host.json` setting `maxConcurrentSessions` to set _target executions per instance_, as in the following example:
149+
131150
```json
132151
{
133152
"version": "2.0",
@@ -138,7 +157,11 @@ For **v5.x+** of the Service Bus extension, modify the `host.json` setting `maxC
138157
}
139158
}
140159
```
141-
For Functions host **v2.x+**, modify the `host.json` setting `maxConcurrentSessions` in `sessionHandlerOptions` to set _target executions per instance_:
160+
161+
# [v2.x+](#tab/v2)
162+
163+
Modify the `host.json` setting `maxConcurrentSessions` in `sessionHandlerOptions` to set _target executions per instance_, as in the following example:
164+
142165
```json
143166
{
144167
"version": "2.0",
@@ -151,10 +174,15 @@ For Functions host **v2.x+**, modify the `host.json` setting `maxConcurrentSessi
151174
}
152175
}
153176
```
177+
---
178+
154179
#### Batch Processing
155-
In this model, each invocation of your function processes a batch of messages.
156180

157-
For **v5.x+** of the Service Bus extension, modify the `host.json` setting `maxMessageBatchSize` to set _target executions per instance_:
181+
In this model, each invocation of your function processes a batch of messages. The specific setting depends on the version of the Service Bus extension.
182+
183+
# [v5.x+](#tab/v5)
184+
185+
Modify the `host.json` setting `maxMessageBatchSize` to set _target executions per instance_, as in the following example:
158186
```json
159187
{
160188
"version": "2.0",
@@ -165,7 +193,11 @@ For **v5.x+** of the Service Bus extension, modify the `host.json` setting `maxM
165193
}
166194
}
167195
```
168-
For Functions host **v2.x+**, modify the `host.json` setting `maxMessageCount` in `batchOptions` to set _target executions per instance_:
196+
197+
# [v2.x+](#tab/v2)
198+
199+
Modify the `host.json` setting `maxMessageCount` in `batchOptions` to set _target executions per instance_, as in the following example:
200+
169201
```json
170202
{
171203
"version": "2.0",
@@ -178,14 +210,21 @@ For Functions host **v2.x+**, modify the `host.json` setting `maxMessageCount` i
178210
}
179211
}
180212
```
213+
---
181214

182215
### Event Hubs
183-
For Event Hubs, Azure Functions scales based on the number of unprocessed events distributed across all the partitions in the hub. By default, the `host.json` attributes used for _target executions per instance_ are `maxEventBatchSize` and `maxBatchSize`. However, if you wish to fine-tune target-based scaling, you can define a separate parameter `targetUnprocessedEventThreshold` that overrides to set _target executions per instance_ without changing the batch settings. If `targetUnprocessedEventThreshold` is set, the total unprocessed event count is divided by this value to determine the number of instances, which is then be rounded up to a worker instance count that creates a balanced partition distribution.
216+
217+
For Azure Event Hubs, Azure Functions scales based on the number of unprocessed events distributed across all the partitions in the event hub. By default, the `host.json` attributes used for _target executions per instance_ are `maxEventBatchSize` and `maxBatchSize`. However, if you choose to fine-tune target-based scaling, you can define a separate parameter `targetUnprocessedEventThreshold` that overrides to set _target executions per instance_ without changing the batch settings. If `targetUnprocessedEventThreshold` is set, the total unprocessed event count is divided by this value to determine the number of instances, which is then be rounded up to a worker instance count that creates a balanced partition distribution.
184218

185219
> [!NOTE]
186-
> Since Event Hubs is a partitioned workload, the target instance count for Event Hubs is capped by the number of partitions in your Event Hub.
220+
> Since Event Hubs is a partitioned workload, the target instance count for Event Hubs is capped by the number of partitions in your event hub.
221+
222+
The specific setting depends on the version of the Event Hubs extension.
223+
224+
# [v5.x+](#tab/v5)
225+
226+
Modify the `host.json` setting `maxEventBatchSize` to set _target executions per instance_, as in the following example:
187227

188-
For **v5.x+** of the Event Hubs extension, modify the `host.json` setting `maxEventBatchSize` to set _target executions per instance_:
189228
```json
190229
{
191230
"version": "2.0",
@@ -197,6 +236,21 @@ For **v5.x+** of the Event Hubs extension, modify the `host.json` setting `maxEv
197236
}
198237
```
199238

239+
When defined in `host.json`, `targetUnprocessedEventThreshold` is used as _target executions per instance_ instead of `maxBatchSize` or `maxEventBatchSize`, as in the following example:
240+
241+
```json
242+
{
243+
"version": "2.0",
244+
"extensions": {
245+
"eventHubs": {
246+
"targetUnprocessedEventThreshold": 23
247+
}
248+
}
249+
}
250+
```
251+
252+
# [v3.x+](#tab/v2)
253+
200254
For **v3.x+** of the Event Hubs extension, modify the `host.json` setting `maxBatchSize` under `eventProcessorOptions` to set _target executions per instance_:
201255
```json
202256
{
@@ -211,7 +265,8 @@ For **v3.x+** of the Event Hubs extension, modify the `host.json` setting `maxBa
211265
}
212266
```
213267

214-
If defined, `targetUnprocessedEventThreshold` in `host.json` will be used as _target executions per instance_ instead of `maxBatchSize` or `maxEventBatchSize`:
268+
When defined in `host.json`, `targetUnprocessedEventThreshold` is used as _target executions per instance_ instead of `maxBatchSize` or `maxEventBatchSize`, as in the following example:
269+
215270
```json
216271
{
217272
"version": "2.0",
@@ -222,9 +277,11 @@ If defined, `targetUnprocessedEventThreshold` in `host.json` will be used as _ta
222277
}
223278
}
224279
```
280+
---
225281

226282
### Storage Queues
227-
For **v2.x**+ of the Storage extension, modify the `host.json` setting `batchSize` to set _target executions per instance_:
283+
284+
For **v2.x+** of the Storage extension, modify the `host.json` setting `batchSize` to set _target executions per instance_:
228285
```json
229286
{
230287
"version": "2.0",
@@ -236,9 +293,14 @@ For **v2.x**+ of the Storage extension, modify the `host.json` setting `batchSiz
236293
}
237294
```
238295

239-
### Cosmos DB
296+
### Azure Cosmos DB
297+
298+
Azure Cosmos DB uses a function-level attribute, `MaxItemsPerInvocation`. The way you set this function-level attribute depends on your function language.
299+
300+
# [C#](#tab/csharp)
301+
302+
For a compiled C# function, set `MaxItemsPerInvocation` in your trigger definition, as shown in the following examples for an in-process C# function:
240303

241-
Cosmos DB uses a function-level attribute, `MaxItemsPerInvocation`. Modify this in `function.json` , or directly in the trigger definition, to set _target executions per instance_:
242304
```C#
243305
namespace CosmosDBSamplesV2
244306
{
@@ -264,7 +326,15 @@ namespace CosmosDBSamplesV2
264326
}
265327

266328
```
267-
Sample `bindings` section of a `function.json` with `MaxItemsPerInvocation` defined:
329+
330+
# [Java](#tab/java)
331+
332+
Java example pending.
333+
334+
# [JavaScript/PowerShell/Python](#tab/node+powershell+python)
335+
336+
For Functions languages that use `function.json`, the `MaxItemsPerInvocation` parameter is defined in the specific binding, as in this Azure Cosmos DB trigger example:
337+
268338
```json
269339
{
270340
"bindings": [
@@ -283,5 +353,10 @@ Sample `bindings` section of a `function.json` with `MaxItemsPerInvocation` defi
283353
]
284354
}
285355
```
356+
357+
Examples for the Python v2 programming model and the JavaScript v4 programming model aren't yet available.
358+
359+
---
360+
286361
> [!NOTE]
287-
> Since Cosmos DB is a partitioned workload, the target instance count for Cosmos DB is capped by the number of physical partitions in your Cosmos DB. For further documentation on Cosmos DB scaling, please see notes on [physical partitions](../cosmos-db/nosql/change-feed-processor.md#dynamic-scaling) and [lease ownership](../cosmos-db/nosql/change-feed-processor.md#dynamic-scaling).
362+
> Since Azure Cosmos DB is a partitioned workload, the target instance count for the database is capped by the number of physical partitions in your container. To learn more about Azure Cosmos DB scaling, see [physical partitions](../cosmos-db/nosql/change-feed-processor.md#dynamic-scaling) and [lease ownership](../cosmos-db/nosql/change-feed-processor.md#dynamic-scaling).

0 commit comments

Comments
 (0)