Skip to content

Commit 531da5f

Browse files
hhunter-mshhunter-ms
committed
merge conflict
Signed-off-by: Hannah Hunter <[email protected]>
2 parents 2dfa700 + 663a61f commit 531da5f

File tree

1 file changed

+94
-26
lines changed

1 file changed

+94
-26
lines changed

‎articles/azure-functions/durable-task-scheduler/durable-task-scheduler-auto-purge.md

Lines changed: 94 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -5,70 +5,138 @@ ms.topic: conceptual
55
ms.date: 04/17/2025
66
---
77

8-
# Set autopurge retention policies for Azure Functions Durable Task Scheduler (preview)
8+
# Set autopurge retention policies for Azure Functions durable task scheduler (preview)
99

10-
Large volumes of completed orchestration instance data can lead to storage bloat, incur higher storage cost, and increase performance issues. The autopurge feature for Durable Task Scheduler offers a lightweight, configurable, and adoptable way to manage orchestration instance clean-up without manual intervention.
10+
Orchestration history data should be purged periodically to free up storage resources. Otherwise, the app will likely observe performance degradation as history data accumulates overtime. The Durable Task Scheduler offers a lightweight, configurable autopurge feature that helps you manage orchestration data clean-up without manual intervention.
1111

12-
Autopurge runs asynchronously in the background to avoid using too many system resources and to prevent blocking or delaying other Durable Task operations. While autopurge doesn't run on a precise schedule, the clean-up rate roughly matches your orchestration scheduling rate.
12+
Autopurge runs asynchronously in the background. It's optimized to avoid using too many system resources to prevent blocking or delaying other Durable Task operations. While autopurge doesn't run on a precise schedule, the clean-up rate roughly matches your orchestration scheduling rate.
1313

1414
## Enable autopurge
1515

16-
Enable autopurge by defining retention policies that control how long to keep completed, failed, or canceled orchestrations. Once enabled, autopurge deletes orchestration instances older than the retention period you set.
16+
Autopurge is an opt-in feature. Enable it by defining retention policies that control how long to keep the data of orchestrations in certain statuses. The autopurge feature covers **completed, failed, canceled, or terminated** statuses only. It does not purge data associated of other statuses such as pending or running. Once enabled, autopurge will periodically delete orchestration instances older than the retention period you set.
1717

18-
You can configure autopurge using:
18+
At the moment, retention policies you define will be applied to **ALL** task hubs in a scheduler.
1919

20-
- Azure Resource Manager
21-
- Azure CLI
22-
- Azure portal
20+
You can define retention policies using:
2321

24-
# [Azure Resource Manager](#tab/arm)
22+
- Azure Resource Manager (ARM)
23+
- Azure CLI
24+
- Azure portal
2525

26-
When configuring in Azure Resource Manager, you can set either a *specific* or a *default* retention policy. Retention value can range from 0 (purge immediately after completion) to any large number.
26+
When configuring autopurge retention policy, you can set either a *specific* or a *default* policy. Retention value can range from 0 (purge immediately after completion) to the maximum integer value, with the unit being **days**. While there is no limit imposed on the max retention period, it's recommended that you don't keep large volumes of stale orchestration data for too long to ensure efficient use of storage resources and maintain app performance.
2727

28-
- **Specific policies** are applied only to the `orchestrationState` specified.
28+
- **Default policy** purges orchestration data *regardless* of `orchestrationState`. The following policy purges orchestration data of all statuses covered by the feature after 2 days:
2929

3030
```json
3131
{
32-
"retentionPeriodInDays": 1,
33-
"orchestrationState": "Completed"
32+
"retentionPeriodInDays": 2
3433
}
3534
```
3635

37-
In this example, the retention policy tells Durable Task Scheduler to keep completed orchestrations for one day, and purge the rest.
38-
39-
- **Default policies** are applied to all purgeable statuses by omitting `orchestrationState`:
36+
- **Specific policy** specifies purging of orchestration data for specific `orchestrationState`. The following policy tells Durable Task Scheduler to keep *completed* orchestration data for 1 day, after which this data is purged.
4037

4138
```json
4239
{
43-
"retentionPeriodInDays": 2
40+
"retentionPeriodInDays": 1,
41+
"orchestrationState": "Completed"
4442
}
4543
```
44+
45+
You can add specific policies to override the default policy that's applied to orchestrations of all statuses. In the example below, the second and third policies override the default policy, so data associated with completed orchestrations is deleted immediately and those associated with failed orchestrations is purged after 60 days. However, because there's no specific policy for orchestrations of statuses canceled and terminated, the default policy still applies so these data are purged after 1 day:
46+
47+
```json
48+
[
49+
{
50+
"retentionPeriodInDays": 1
51+
},
52+
{
53+
"retentionPeriodInDays": 0,
54+
"orchestrationState": "Completed"
55+
},
56+
{
57+
"retentionPeriodInDays": 60,
58+
"orchestrationState": "Failed"
59+
}
60+
]
61+
```
4662

4763
[For more information, see the API reference spec for Durable Task Scheduler retention policies.](/rest/api/durabletask/retention-policies/create-or-replace?view=rest-durabletask-2025-04-01-preview&preserve-view=true)
4864

65+
# [Azure Resource Manager](#tab/arm)
66+
4967
# [Azure CLI](#tab/cli)
50-
Need
68+
Create or update the retention policy by running the following:
69+
70+
```azurecli
71+
az rest --method put --url "/subscriptions/SUBSCRIPTION_ID/resourceGroups/RESOURCE_GROUP/providers/Microsoft.DurableTask/schedulers/SCHEDULER_NAME/retentionPolicies/default?api-version=2025-04-01-preview" --body '{
72+
"name": "default",
73+
"type": "private.durabletask/schedulers/retentionPolicies",
74+
"properties": {
75+
"retentionPolicies": [
76+
{
77+
"retentionPeriodInDays": 1
78+
},
79+
{
80+
"retentionPeriodInDays": 0,
81+
"orchestrationState": "Completed"
82+
},
83+
{
84+
"retentionPeriodInDays": 60,
85+
"orchestrationState": "Failed"
86+
},
87+
]
88+
}
89+
}'
90+
```
91+
92+
The following is returned upon successful creation:
93+
```json
94+
{
95+
"id": "/subscriptions/SUBSCRIPTION_ID/resourceGroups/RESOURCE_GROUP_NAME/providers/Microsoft.DurableTask/schedulers/SCHEDULER_NAMER/retentionPolicies/default",
96+
"name": "default",
97+
"properties": {
98+
"provisioningState": "Succeeded",
99+
"retentionPolicies": [
100+
{
101+
"orchestrationState": "Completed",
102+
"retentionPeriodInDays": 0
103+
},
104+
{
105+
"orchestrationState": "Failed",
106+
"retentionPeriodInDays": 30
107+
}
108+
]
109+
},
110+
"systemData": {
111+
"createdAt": "2025-04-23T23:41:17.3165122Z",
112+
"createdBy": "[email protected]",
113+
"createdByType": "User",
114+
"lastModifiedAt": "2025-04-23T23:41:17.3165122Z",
115+
"lastModifiedBy": "[email protected]",
116+
"lastModifiedByType": "User"
117+
},
118+
"type": "microsoft.durabletask/schedulers/retentionpolicies"
119+
}
120+
```
51121

52122
# [Azure portal](#tab/portal)
53123
Need
54124

55125
---
56126

57-
> [!NOTE]
58-
> If both a specific and a default policy are set, the specific policy takes priority.
59-
60127
## Disable autopurge
61-
62128
# [Azure Resource Manager](#tab/arm)
63129

64-
To disable autopurge retention policies, just delete the policy from the template. Durable Task Scheduler automatically stops cleaning up instances.
65-
66130
# [Azure CLI](#tab/cli)
131+
To disable autopurge, simply delete the retention policies. The Durable Task Scheduler will stop cleaning orchestration data within 5 to 10 minutes.
67132
Need
68133

69-
# [Azure portal](#tab/portal)
70-
Need
134+
```azurecli
135+
az rest --method delete --url "/subscriptions/SUBSCRIPTION_ID/resourceGroups/RESOURCE_GROUP_NAME/providers/Microsoft.DurableTask/schedulers/SCHEDULER_NAMER/retentionPolicies/default?api-version=2025-04-01-preview"
136+
```
71137

138+
# [Azure portal](#tab/arm)
139+
Experience coming soon!
72140
---
73141

74142

0 commit comments

Comments
 (0)