You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/azure-functions/durable-task-scheduler/durable-task-scheduler-auto-purge.md
+37-22Lines changed: 37 additions & 22 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,22 +7,25 @@ ms.date: 04/17/2025
7
7
8
8
# Set autopurge retention policies for Azure Functions durable task scheduler (preview)
9
9
10
-
Orchestration history data should be purged periodically to free up storage resources. Otherwise, the app will likely observe performance degradation as history data accumulates overtime. The Durable Task Scheduler offers a lightweight, configurable autopurge feature to help you manage orchestration data clean-up without manual intervention.
10
+
Orchestration history data should be purged periodically to free up storage resources. Otherwise, the app will likely observe performance degradation as history data accumulates overtime. The Durable Task Scheduler offers a lightweight, configurable autopurge feature that helps you manage orchestration data clean-up without manual intervention.
11
11
12
-
Autopurge runs asynchronously in the background and is optimized to avoid using too many system resources, so it won't prevent blocking or delaying other Durable Task operations. While autopurge doesn't run on a precise schedule, the clean-up rate roughly matches your orchestration scheduling rate.
12
+
Autopurge runs asynchronously in the background. It's optimized to avoid using too many system resources to prevent blocking or delaying other Durable Task operations. While autopurge doesn't run on a precise schedule, the clean-up rate roughly matches your orchestration scheduling rate.
13
13
14
14
## Enable autopurge
15
15
16
-
Autopurge is an opt-in feature. Enable it by defining retention policies that control how long to keep completed, failed, or canceled orchestrations. Once enabled, autopurge will periodically delete orchestration instances older than the retention period you set.
16
+
Autopurge is an opt-in feature. Enable it by defining retention policies that control how long to keep the data of orchestrations in certain statuses. The autopurge feature covers **completed, failed, canceled, or terminated** statuses only. It does not purge data associated of other statuses such as pending or running. Once enabled, autopurge will periodically delete orchestration instances older than the retention period you set.
17
17
18
-
You can configure autopurge using:
18
+
At the moment, retention policies you define will be applied to **ALL** task hubs in a scheduler.
19
19
20
-
- Azure Resource Manager API
20
+
You can define retention policies using:
21
+
22
+
- Azure Resource Manager (ARM)
21
23
- Azure CLI
24
+
- Azure portal
22
25
23
-
When configuring through retention policy, you can set either a *specific* or a *default* policy. Retention value can range from 0 (purge immediately after completion) to the maximum integer value. While there is no limit imposed on the max retention period, it is recommended that you keep completed orchestration data only for as long as you need it to free up storage resources and ensure app performance.
26
+
When configuring autopurge retention policy, you can set either a *specific* or a *default* policy. Retention value can range from 0 (purge immediately after completion) to the maximum integer value, with the unit being **days**. While there is no limit imposed on the max retention period, it's recommended that you don't keep large volumes of stale orchestration data for too long to ensure efficient use of storage resources and maintain app performance.
24
27
25
-
-**Default policy** purges orchestration data *regardless* of `orchestrationState`. The following policy purges orchestration data of all statuses after 2 days:
28
+
-**Default policy** purges orchestration data *regardless* of `orchestrationState`. The following policy purges orchestration data of all statuses covered by the feature after 2 days:
26
29
27
30
```json
28
31
{
@@ -38,22 +41,30 @@ When configuring through retention policy, you can set either a *specific* or a
38
41
"orchestrationState": "Completed"
39
42
}
40
43
```
41
-
42
-
If you want to override a specific policy, you can add another specific policy. In the example below. the second specific policy would override the first, so the data of *failed* orchestrations is purged after 2 days:
44
+
45
+
You can add specific policies to override the default which applies to orchestrations of all statuses. In the example below, the second and third policies override the default, so data associated with completed orchestrations is deleted immediately and those associated with failed orchestrations is purged after 60 days. However, because there's no specific policy for orchestrations of statuses canceled and terminated, the default policy still applies so these data are purged after 1 day:
43
46
44
47
```json
48
+
[
49
+
{
50
+
"retentionPeriodInDays": 1
51
+
},
52
+
{
53
+
"retentionPeriodInDays": 0,
54
+
"orchestrationState": "Completed"
55
+
},
45
56
{
46
-
"retentionPeriodInDays": 2,
57
+
"retentionPeriodInDays": 60,
47
58
"orchestrationState": "Failed"
48
59
}
60
+
]
49
61
```
50
62
51
63
[For more information, see the API reference spec for Durable Task Scheduler retention policies.](/rest/api/durabletask/retention-policies/create-or-replace?view=rest-durabletask-2025-04-01-preview&preserve-view=true)
52
64
53
65
# [Azure Resource Manager](#tab/arm)
54
66
55
67
# [Azure CLI](#tab/cli)
56
-
57
68
Set the retention policy by running the following:
58
69
59
70
```azurecli
@@ -66,35 +77,39 @@ Set the retention policy by running the following:
66
77
"retentionPeriodInDays": 1
67
78
},
68
79
{
69
-
"retentionPeriodInDays": 1,
80
+
"retentionPeriodInDays": 0,
70
81
"orchestrationState": "Completed"
71
82
},
72
83
{
73
-
"retentionPeriodInDays": 2,
84
+
"retentionPeriodInDays": 60,
74
85
"orchestrationState": "Failed"
75
86
},
76
87
]
77
88
}
78
89
}'
79
90
```
80
91
92
+
# [Azure portal](#tab/portal)
93
+
Experience coming soon!
81
94
---
82
95
83
-
> [!NOTE]
84
-
> If both a specific and a default policy are set, the specific policy takes priority.
85
-
86
96
## Disable autopurge
87
-
88
97
# [Azure Resource Manager](#tab/arm)
89
98
90
-
To disable autopurge retention policies, just delete the policy from the template. Durable Task Scheduler automatically stops cleaning up instances.
91
-
92
99
# [Azure CLI](#tab/cli)
93
-
todo
100
+
To disable autopurge, run the following with the `retentionPolicies` property removed. The Durable Task Scheduler automatically stops cleaning up instances.
94
101
95
-
# [Azure portal](#tab/portal)
96
-
todo
102
+
```azurecli
103
+
az rest --method put --url "/subscriptions/SUBSCRIPTION_ID/resourceGroups/RESOURCE_GROUP/providers/Microsoft.DurableTask/schedulers/SCHEDULER_NAME/retentionPolicies/default?api-version=2025-04-01-preview" --body '{
0 commit comments