You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
title: Set autopurge retention policies for Azure Functions Durable Task Scheduler (preview)
3
3
description: Learn about how and why you'd want to configure autopurge retention policies for Durable Task Scheduler.
4
4
ms.topic: conceptual
5
-
ms.date: 04/17/2025
5
+
ms.date: 04/28/2025
6
6
---
7
7
8
-
# Set autopurge retention policies for Azure Functions durable task scheduler (preview)
8
+
# Set autopurge retention policies for Azure Functions Durable Task Scheduler (preview)
9
9
10
10
Orchestration history data should be purged periodically to free up storage resources. Otherwise, the app will likely observe performance degradation as history data accumulates overtime. The Durable Task Scheduler offers a lightweight, configurable autopurge feature that helps you manage orchestration data clean-up without manual intervention.
11
11
12
-
Autopurge runs asynchronously in the background. It's optimized to avoid using too many system resources to prevent blocking or delaying other Durable Task operations. While autopurge doesn't run on a precise schedule, the clean-up rate roughly matches your orchestration scheduling rate.
12
+
Autopurge operates asynchronously in the background, optimized to minimize system resource usage and prevent interference with other Durable Task operations. Although autopurge doesn't adhere to a strict schedule, its clean-up rate generally aligns with your orchestration scheduling rate.
13
13
14
-
## Enable autopurge
14
+
## How it works
15
15
16
-
Autopurge is an opt-in feature. Enable it by defining retention policies that control how long to keep the data of orchestrations in certain statuses. The autopurge feature covers **completed, failed, canceled, or terminated** statuses only. It does not purge data associated of other statuses such as pending or running. Once enabled, autopurge will periodically delete orchestration instances older than the retention period you set.
16
+
Autopurge is an opt-in feature. Enable it by defining retention policies that control how long to keep the data of orchestrations in certain statuses. The autopurge feature purges data associated only with the following statuses:
17
+
-`Completed`
18
+
-`Failed`
19
+
-`Canceled`
20
+
-`Terminated`
17
21
18
-
At the moment, retention policies you define will be applied to **ALL** task hubs in a scheduler.
22
+
Autopurge ignores data associated with the following statuses:
23
+
-`Pending`
24
+
-`Running`
19
25
20
-
You can define retention policies using:
26
+
[Once enabled,](#enable-autopurge) autopurge periodically deletes orchestration instances older than the retention period you set.
21
27
22
-
- Azure Resource Manager (ARM)
23
-
- Azure CLI
24
-
- Azure portal
28
+
> [!NOTE]
29
+
> Retention policies you define are applied to **all** task hubs in a scheduler.
30
+
31
+
### Policy value
25
32
26
-
When configuring autopurge retention policy, you can set either a *specific* or a *default* policy. Retention value can range from 0 (purge immediately after completion) to the maximum integer value, with the unit being **days**. While there is no limit imposed on the max retention period, it's recommended that you don't keep large volumes of stale orchestration data for too long to ensure efficient use of storage resources and maintain app performance.
33
+
Retention value can range from 0 (purge immediately after completion) to the maximum integer value, with the unit being **days**.
34
+
35
+
Although retention periods have no maximum limit, we recommend you avoid retaining large volumes of stale orchestration data for extended periods. This practice ensures efficient use of storage resources and maintains optimal app performance.
36
+
37
+
### Types of policies
38
+
39
+
When configuring an autopurge retention policy, you can set either a *specific* or a *default* policy.
27
40
28
41
-**Default policy** purges orchestration data *regardless* of `orchestrationState`. The following policy purges orchestration data of all statuses covered by the feature after 2 days:
29
42
@@ -33,7 +46,7 @@ When configuring autopurge retention policy, you can set either a *specific* or
33
46
}
34
47
```
35
48
36
-
- **Specific policy** specifies purging of orchestration data for specific `orchestrationState`. The following policy tells Durable Task Scheduler to keep *completed* orchestration data for 1 day, after which this data is purged.
49
+
- **Specific policy** defines purging of orchestration data for specific `orchestrationState`. The following policy tells Durable Task Scheduler to keep *completed* orchestration data for 1 day, after which this data is purged.
37
50
38
51
```json
39
52
{
@@ -42,7 +55,11 @@ When configuring autopurge retention policy, you can set either a *specific* or
42
55
}
43
56
```
44
57
45
-
You can add specific policies to override the default policy that's applied to orchestrations of all statuses. In the example below, the second and third policies override the default policy, so data associated with completed orchestrations is deleted immediately and those associated with failed orchestrations is purged after 60 days. However, because there's no specific policy for orchestrations of statuses canceled and terminated, the default policy still applies so these data are purged after 1 day:
58
+
Add specific policies to override the default policy applied to orchestrations, regardless of status. In the example below, the second and third policies override the default policy (`"retentionPeriodInDays": 1`).
59
+
- Data associated with completed orchestrations is deleted immediately.
60
+
- Data associated with failed orchestrations is purged after 60 days.
61
+
62
+
However, since no specific policy is set for canceled or terminated orchestrations, the default policy still applies to them, purging their data after 1 day.
46
63
47
64
```json
48
65
[
@@ -62,9 +79,104 @@ You can add specific policies to override the default policy that's applied to o
62
79
63
80
[For more information, see the API reference spec for Durable Task Scheduler retention policies.](/rest/api/durabletask/retention-policies/create-or-replace?view=rest-durabletask-2025-04-01-preview&preserve-view=true)
64
81
82
+
## Enable autopurge
83
+
84
+
You can define retention policies using:
85
+
86
+
- Bicep
87
+
- Azure Resource Manager (ARM)
88
+
- Azure CLI
89
+
90
+
# [Bicep](#tab/bicep)
91
+
92
+
You can create or update retention policies by adding the `retentionPolicies` configuration to your Bicep file. Make sure you're pulling from the latest preview version.
You can create or update retention policies using the Azure Resource Manager API using the following request. Make sure you're pulling from the latest preview version.
119
+
120
+
```HTTP
121
+
PUT https://management.azure.com/subscriptions/SUBSCRIPTION_ID/resourceGroups/RESOURCE_GROUP/providers/Microsoft.DurableTask/schedulers/SCHEDULER_NAME/retentionPolicies/default?api-version=2025-04-01-preview
122
+
123
+
{
124
+
"properties": {
125
+
"retentionPolicies": [
126
+
{
127
+
"retentionPeriodInDays": 1
128
+
},
129
+
{
130
+
"retentionPeriodInDays": 0,
131
+
"orchestrationState": "Completed"
132
+
},
133
+
{
134
+
"retentionPeriodInDays": 60,
135
+
"orchestrationState": "Failed"
136
+
}
137
+
]
138
+
}
139
+
}
140
+
```
141
+
142
+
**Example response**
143
+
144
+
If creation is successful, you'll see the following response.
0 commit comments