|
1 | 1 | ---
|
2 |
| -title: Choosing your orchestration framework |
3 |
| -description: This article provides guidance to help developers pick an orchestration framework. |
| 2 | +title: Choosing an orchestration framework |
| 3 | +description: Learn which orchestration framework works for your scenario. |
4 | 4 | ms.topic: conceptual
|
5 |
| -ms.date: 04/28/2025 |
| 5 | +ms.date: 05/05/2025 |
6 | 6 | ---
|
7 | 7 |
|
8 | 8 | # Choosing an orchestration framework
|
9 | 9 |
|
10 |
| -The goal of this article is to provide information about two developer-oriented orchestration frameworks supported by Azure and considerations for when to choose which. The article also provides some context around scenarios that warrant the use of an orchestration framework. |
| 10 | +Azure offers two developer-oriented orchestration frameworks you can use to build apps: **Durable Task SDKs** and **Durable Functions**. In this article, you learn: |
| 11 | + |
| 12 | +> [!div class="checklist"] |
| 13 | +> - The benefits of using an orchestration framework. |
| 14 | +> - Which framework works best for your scenario. |
11 | 15 |
|
12 | 16 | ## Scenarios requiring orchestration
|
13 |
| -Developers often find themselves creating solutions that involve multi-step operations, some of which may be long-running. These multiple-step operations are referred to as *orchestrations* or *workflows*. For example, an e-commerce website would need an order processing workflow, which has steps (implemented as microservices) that need to happen sequentially to: |
| 17 | + |
| 18 | +Application or orchestration patterns that benefit from the statefulness offered by an orchestration framework include: |
| 19 | +- **Function chaining:** For executing sequential workflow steps in order, passing data between steps with data transformations at each step, and building pipelines where each activity builds on the previous one. |
| 20 | +- **Fan-out/fan-in:** For batch jobs, ETL (extract, transfer, and load), and any scenario that requires parallel processing. |
| 21 | +- **Human interactions:** For two-factor authentication, workflows that require human approval. |
| 22 | +- **Asynchronous HTTP APIs:** For any scenario where a client doesn't want to wait for long-running tasks to complete. |
| 23 | + |
| 24 | +The two following scenarios share the commonly used *function chaining* pattern. |
| 25 | + |
| 26 | +### Processing orders on an e-commerce website |
| 27 | + |
| 28 | +Let's say you create an e-commerce solution that involves multi-step operations, some of which may be long-running. These multiple-step operations are referred to as *orchestrations* or *workflows*. Your website needs an order processing workflow that includes steps happening sequentially to: |
14 | 29 | - Check the inventory
|
15 | 30 | - Send order confirmation
|
16 | 31 | - Process payment
|
17 | 32 | - Generate invoice
|
18 | 33 |
|
19 |
| -Another example would be an intelligent trip planner. The planner might suggest ideas based on user requirements, get preference confirmation, and then make required bookings. You can implement an agent for each task and orchestrate the steps for invoking these agents as a workflow. |
20 |
| - |
21 |
| -Since orchestrations involve multiple or long-running steps, it's important that the framework tracks the application state so that execution can continue from the point of failure rather than the beginning. One way to ensure this statefulness is to continuously checkpoint orchestration states to a persistence layer while it runs. The next section demonstrates how orchestration frameworks maintain statefulness while also taking care automatic retries while you orchestrate your workflows without the burden of architecting for fault tolerance. |
| 34 | +### Invoking AI agents for planning a trip |
22 | 35 |
|
23 |
| -The two previous scenarios share a commonly used orchestration pattern, which is *sequential chaining* of operations. There are other patterns that benefit from the statefulness of an orchestration framework: |
24 |
| -- *Fan-out/fan-in*: For batch jobs, ETL (extract, transfer, and load), and any scenario that requires parallel processing |
25 |
| -- *Human interactions*: For two-factor authentication, workflows that require human approval |
26 |
| -- *Asynchronous HTTP APIs*: For any scenario where a client doesn't want to wait for long-running tasks to complete |
| 36 | +In this scenario, you created an intelligent trip planner. The planner: |
| 37 | +- Suggests ideas based on user requirements |
| 38 | +- Gets preference confirmation |
| 39 | +- Makes required bookings |
27 | 40 |
|
28 |
| -The next section covers the two orchestration frameworks supported by Azure: **Durable Functions** and **Durable Task SDKs (currently in public preview)**. |
| 41 | +You can implement an agent for each task and orchestrate the steps for agent invocation as a workflow. |
29 | 42 |
|
30 | 43 | ## Orchestration framework options
|
31 |
| -Both orchestrations frameworks described in this section are designed for developers, so they're code-centric and are available in various popular programming languages. |
32 | 44 |
|
33 |
| -### Durable Functions |
| 45 | +Since orchestrations involve multiple or long-running steps, it's important for orchestration frameworks to maintain application statefulness so that execution can continue from the point of failure rather than the beginning. One way to ensure this statefulness is to continuously checkpoint orchestration states to a persistence layer while it runs. |
| 46 | + |
| 47 | +Both Durable Functions and the Durable Task SDKs maintain statefulness while also handling automatic retries while you orchestrate your workflows without the burden of architecting for fault tolerance. |
| 48 | + |
| 49 | +> [!NOTE] |
| 50 | +> Durable Functions and the Durable Task SDKs are code-centric and available in various popular programming languages. |
| 51 | +
|
| 52 | +### Durable Functions |
| 53 | + |
34 | 54 | As a feature of Azure Functions, [Durable Functions](../durable-functions-overview.md) inherits numerous assets, such as:
|
35 | 55 | - Integrations with other Azure services through the Functions extensions
|
36 | 56 | - Local development experience
|
37 | 57 | - Serverless pricing model, and more.
|
38 | 58 |
|
39 |
| -Other than running on the Functions platform, Durable Functions apps can also be run on Azure App Service and Azure Container App just like a regular Function app. |
| 59 | +Aside from running on the Functions platform, Durable Functions apps can also be run on Azure App Service and Azure Container Apps. |
40 | 60 |
|
41 |
| -Durable Functions has a special feature called [Durable Entities](../durable-functions-entities.md), which are similar in concept to virtual actors or grains in the Orleans framework. Entities allow you to keep small pieces of states for objects in a distributed system. For example, you could use entities to model users of a microblogging app or the counter of an exercise app. Learn more about Durable Entities. |
| 61 | +Durable Functions includes a special feature called [Durable Entities](../durable-functions-entities.md), which are similar in concept to virtual actors or grains in the Orleans framework. *Entities* allow you to keep small pieces of states for objects in a distributed system. For example, you could use entities to model users of a microblogging app or the counter of an exercise app. |
42 | 62 |
|
43 |
| -As mentioned earlier, orchestration frameworks need to persist state information. In the context of Durable Functions, states are persisted in something called a [storage backend](../durable-functions-storage-providers.md). Durable Functions supports two "bring-your-own" (BYO) backends - Azure Storage and Microsoft SQL - and an Azure managed backend called Durable Task Scheduler (more information about the scheduler in the next section). |
| 63 | +Durable Functions persists states in a [storage backend](../durable-functions-storage-providers.md). Durable Functions supports: |
| 64 | + |
| 65 | +- Two "bring-your-own" (BYO) backends: |
| 66 | + - Azure Storage |
| 67 | + - Microsoft SQL |
| 68 | +- An Azure managed backend: |
| 69 | + - [Durable Task Scheduler](#durable-task-sdks-with-durable-task-scheduler-preview) |
44 | 70 |
|
45 | 71 | #### When to use Durable Functions
|
46 |
| -If you need to build event-driven apps with workflows, the recommendation would be to consider Durable Functions. The Azure Functions extensions that provide integrations with other Azure services makes building event-driven scenarios easy. For example, if you need to start a Durable Functions orchestration when a message comes into your Azure Service Bus or a file uploads to Azure Blob Storage, you can easily implement that with Durable Functions. Or if you need an orchestration to start periodically or in reaction to an HTTP request, then you can build that with the Azure Functions timer and HTTP trigger, respectively. |
47 | 72 |
|
48 |
| -Another scenario for choosing Durable Functions would be when your solution benefits from using Durable Entities, since the Durable Task SDKs don't provide this feature. |
| 73 | +Consider using Durable Functions if you need to build event-driven apps with workflows. The Azure Functions extensions that provide integrations with other Azure services makes building event-driven scenarios easy. For example, with Durable Functions: |
| 74 | + |
| 75 | +- You can easily start a Durable Functions orchestration when a message comes into your Azure Service Bus or a file uploads to Azure Blob Storage. |
| 76 | +- You can easily build an orchestration to start periodically or in reaction to an HTTP request with the Azure Functions timer and HTTP trigger, respectively. |
| 77 | +- Your solution benefits from using Durable Entities, since the Durable Task SDKs don't provide this feature. |
| 78 | +- You're already writing Azure Function apps and realized that you need workflow. Since its programming model is similar to Functions, you can accelerate your developer with Durable Functions. |
49 | 79 |
|
50 |
| -Lastly, if you're already writing Azure Function apps and realized that you need workflow, you probably want to use Durable Functions. Its programming model is very similar to Functions. You can accelerate your developer in this case. |
| 80 | +#### Try it out |
51 | 81 |
|
52 |
| -**Quickstarts** |
53 |
| -- Create a durable functions app with Azure Storage backend |
54 |
| - - [.NET](../durable-functions-isolated-create-first-csharp.md) |
55 |
| - - [Python](../quickstart-python-vscode.md) |
56 |
| - - [JavaScript/TypeScript](../quickstart-js-vscode.md) |
57 |
| - - [Java](../quickstart-java.md) |
58 |
| - - [PowerShell](../quickstart-powershell-vscode.md) |
59 |
| -- [Configure a durable functions app with Durable Task Scheduler](./quickstart-durable-task-scheduler.md) |
60 |
| -- [Configure a durable functions app with MSSQL](../quickstart-mssql.md) |
| 82 | +Walk through one of the following quickstarts or scenarios to configure your function apps with one of the storage backends available for Durable Functions. |
61 | 83 |
|
62 |
| -**Scenario samples** |
63 |
| -- Order processing workflow - [.NET](/samples/azure-samples/durable-functions-order-processing/durable-func-order-processing/), [Python][TODO] |
64 |
| -- Intelligent PDF summarizer - [.NET][TODO], [Python][TODO] |
| 84 | +##### Quickstarts |
65 | 85 |
|
66 |
| -### Durable Task SDKs with Durable Task Scheduler (public preview) |
67 |
| -The Durable Task SDKs are client SDKs that allows the orchestrations you write to connect to the orchestration engine in Azure. Apps that use the Durable Task SDKs can be run on any compute platforms, including: |
| 86 | +| | Quickstart | Description | |
| 87 | +| - | ---------- | ----------- | |
| 88 | +| **Durable Task Scheduler (preview)** | [Configure a durable functions app with Durable Task Scheduler](./quickstart-durable-task-scheduler.md) | Configure a "hello world" Durable Functions app to use the Durable Task Scheduler as the backend storage provider, test locally, and publish to Azure. | |
| 89 | +| **Azure Storage** | Create a durable functions app with Azure Storage backend:<br>- [.NET](../durable-functions-isolated-create-first-csharp.md)<br>- [Python](../quickstart-python-vscode.md)<br>- [JavaScript/TypeScript](../quickstart-js-vscode.md)<br>- [Java](../quickstart-java.md)<br>- [PowerShell](../quickstart-powershell-vscode.md) | Locally create and test a "hello world" Durable Functions app and deploy to Azure. | |
| 90 | +| **MSSQL** | [Configure a durable functions app with MSSQL](../quickstart-mssql.md) | Using an existing "hello world" Durable Functions app, configure the MSSQL backend storage provider, test locally, and publish to Azure. | |
| 91 | + |
| 92 | +##### Samples |
| 93 | + |
| 94 | +| | Sample | Description | |
| 95 | +| - | ---------- | ----------- | |
| 96 | +| **Order processing workflow** | Create an order processing workflow with Durable Functions with Azure Storage:<br>- [.NET](/samples/azure-samples/durable-functions-order-processing/durable-func-order-processing/)<br>- [Python](/samples/azure-samples/durable-functions-order-processing-python/durable-func-order-processing-py/) | Configure a "hello world" Durable Functions app to use the Durable Task Scheduler as the backend storage provider, test locally, and publish to Azure. | |
| 97 | +| **Intelligent PDF summarizer** | Create an intelligent application using Azure Durable Functions, Azure Storage, Azure Developer CLI, and more:<br>- [.NET](/samples/azure-samples/intelligent-pdf-summarizer-dotnet/durable-func-pdf-summarizer-csharp/)<br>- [Python](/samples/azure-samples/intelligent-pdf-summarizer/durable-func-pdf-summarizer/) | Locally create and test a "hello world" Durable Functions app and deploy to Azure. | |
| 98 | + |
| 99 | +### Durable Task SDKs with Durable Task Scheduler (preview) |
| 100 | + |
| 101 | +The Durable Task SDKs are client SDKs that must be used with the Durable Task Scheduler. The Durable Task SDKs connect the orchestrations you write to the Durable Task Scheduler orchestration engine in Azure. Apps that use the Durable Task SDKs can be run on any compute platform, including: |
68 | 102 | - Azure Kubernetes Service
|
69 | 103 | - Azure Container Apps
|
70 | 104 | - Azure App Service
|
71 |
| -- VMs on-premises |
| 105 | +- Virtual Machines (VMs) on-premises |
72 | 106 |
|
73 |
| -The Durable Task SDKs must be used with the Durable Task Scheduler (currently in public preview), where the latter plays both the role of the orchestration engine and the storage backend for orchestration state persistence. As mentioned previously, the Azure-managed Durable Task Scheduler eliminates management overhead. The scheduler is also optimized to provide high orchestration throughput, offers an out-of-the-box dashboard for orchestrations monitoring and debugging, as well as a local emulator and more. Learn more about the [Durable Task Scheduler](./durable-task-scheduler.md). |
| 107 | +The [Durable Task Scheduler](./durable-task-scheduler.md) (currently in preview) plays the role of both the orchestration engine and the storage backend for orchestration state persistence. The Azure-managed Durable Task Scheduler: |
| 108 | +- Eliminates management overhead |
| 109 | +- Provides high orchestration throughput |
| 110 | +- Offers an out-of-the-box dashboard for orchestration monitoring and debugging |
| 111 | +- Includes a local emulator |
74 | 112 |
|
75 | 113 | #### When to use Durable Task SDKs
|
76 |
| -Use the Durable Task SDKs when your app needs only workflows. The Durable Task SDKs provide a lightweight and relatively un-opinionated programming model for authoring workflows. |
77 | 114 |
|
78 |
| -The other reason for using the Durable Task SDKs is when you need to run apps on Azure Kubernetes Services or VMs on-premises with official Microsoft support. While Durable Functions can be run on these platforms as well, there's no official support. |
| 115 | +If your app only needs workflows, the Durable Task SDKs provide a lightweight and relatively unopinionated programming model for authoring workflows. |
79 | 116 |
|
80 |
| -<!-- uncomment in quickstart PR |
81 |
| -**Quickstarts** |
82 |
| -- [.NET][TODO] |
83 |
| -- [Python][TODO] |
84 |
| -- [Java][TODO] |
85 |
| ---> |
| 117 | +When you need to run apps on Azure Kubernetes Services or VMs on-premises with official Microsoft support. While Durable Functions can be run on these platforms as well, there's no official support. |
86 | 118 |
|
87 |
| -> [!NOTE] |
88 |
| -> The Durable Task Framework is an open-source .NET orchestration framework. Like the Durable Task SDKs, it can be used to build apps that run on platforms like the Azure Kubernetes Services. However, we don't recommend new customers to use Durable Task Framework because Microsoft doesn't officially support it. |
| 119 | +#### Try it out |
89 | 120 |
|
| 121 | +Walk through one of the following quickstarts to configure your applications to use the Durable Task Scheduler with the Durable Task SDKs. |
90 | 122 |
|
| 123 | +| | Quickstart | Description | |
| 124 | +| - | ---------- | ----------- | |
| 125 | +| **Local development quickstart** | [Create an app with Durable Task SDKs and Durable Task Scheduler](./quickstart-portable-durable-task-sdks.md) using either the .NET, Python, or Java SDKs. | Run a fan-in/fan-out orchestration locally using the Durable Task Scheduler emulator and review orchestration history using the dashboard. | |
| 126 | +| **Deploy to Azure Container Apps using Azure Developer CLI** | [Configure Durable Task SDKs in your container app with Azure Functions Durable Task Scheduler][./quickstart-container-apps-durable-task-sdk.md] | Deploy a function chaining pattern solution using the Azure Developer CLI. | |
91 | 127 |
|
| 128 | +## Limitations |
| 129 | + |
| 130 | +- **The Durable Task Framework (DTFx) support** |
| 131 | + - DTFx is an open-source .NET orchestration framework similar to the .NET Durable Task SDK. While it *can* be used to build apps that run on platforms like Azure Kubernetes Services, **DTFx doesn't receive official Microsoft support**. |
92 | 132 |
|
| 133 | +## Next steps |
93 | 134 |
|
| 135 | +> [!div class="nextstepaction"] |
| 136 | +> [Learn more about the Durable Task Scheduler](./durable-task-scheduler.md) |
0 commit comments