Significant updates to main article

Author: cgillum

articles/azure-functions/durable/durable-task-scheduler/durable-task-scheduler.md

@@ -10,46 +10,85 @@ ms.subservice: durable
 
 # Azure Functions durable task scheduler backend for durable functions (preview)
 
-The Azure Functions durable task scheduler is a fully managed backend for durable function apps. It's built from scratch and provides several key benefits not available in the "bring your own (BYO)" backend options today. This document discusses different aspects of the managed backend including architecture, concepts, and key features.
+The Azure Functions durable task scheduler is a fully managed backend for durable function apps. It provides several key benefits not available in the "bring your own (BYO)" backend options for durable functions today. This document discusses different aspects of the managed backend including architecture, concepts, and key features.
 
 For an in-depth comparison between the supported storage providers for durable function apps, see the [storage providers](../durable-functions-storage-providers.md) documentation.
 
 ## Architecture
 
-The durable task scheduler is designed from the ground up to be the fastest and most efficient backend for durable functions. The scheduler resource has its own dedicated compute and memory resources optimized for:
+Instances of the durable task scheduler can be created using Azure Resource Manager and are of type [Microsoft.DurableTask/scheduler](microsoft.durabletask/schedulers). Each *scheduler* resource internally has its own dedicated compute and memory resources optimized for:
 
 - Dispatching orchestrator, activity, and entity work items
 - Storing and querying history at scale with minimal latency
-- Providing a rich monitoring experience through the durable task scheduler dashboard
+- Providing a rich monitoring experience through the durable task scheduler dashboard.
+
+Unlike the BYO storage providers, the durable task scheduler provider is a purpose-built backend-as-a-service optimized for the specific needs of the Durable Task Framework.
+
+The following diagram shows the architecture of the durable task scheduler backend and its interaction with connected apps.
 
 :::image type="content" source="media/durable-task-scheduler/architecture.png" alt-text="Diagram of the durable task scheduler architecture.":::
 
-Unlike other generic storage providers, durable task scheduler is a purpose-built backend-as-a-service optimized for the specific needs of the Durable Task Framework. Durable task scheduler does the heavy lifting of work item dispatching, partition management, and other tasks typically handled internally by the BYO backends running in the app process (like the Azure Functions host). As a result, durable apps using durable task scheduler run with less overhead.
+### Separation of concerns
 
-## Feature highlight
+The durable task scheduler runs in Azure as a separate resource from the app. This separation allows the scheduler to scale independently of the app and provides better isolation between the two components. This isolation is important for several reasons:
 
-> [!NOTE]
-> The following features are available at the current stage of public preview with more under way. To report problems or request new features, submit an issue in the [durable task scheduler repository](https://aka.ms/dts-samples). 
+- **Reduced resource consumption**: BYO storage providers can consume a significant amount of CPU and memory resources. This resource consumption is due to the overhead of managing partitions and other complex state store interactions. By using a managed scheduler instead of a BYO storage provider, your app instances can run more efficiently and with less resource contention.
+
+- **Fault isolation**: Stability or availability issues in the durable task scheduler will not affect the stability or availability of your connected apps. With BYO storage providers, instability in the backend provider (which is a complex component) can create instability in the app logic. By separating the scheduler from the app, you can reduce the risk of cascading failures and improve overall reliability.
+
+- **Independent scaling**: The scheduler resource can be scaled independently of the app, allowing for better infrastructure resource management and cost optimization. For example, multiple apps can share the same scheduler resource, improving overall resource utilization. This capability is especially useful for organizations with multiple teams or projects that require durable functions.
+
+- **Improved support experience**: The durable task scheduler is a managed service, which means that Azure can provide better support and diagnostics for issues related to the scheduler. When using a BYO storage provider, you might need to troubleshoot issues related to the backend provider, which can be complex and time-consuming. A managed service allows Azure to take care of the underlying infrastructure and provide a more streamlined support experience.
+
+### App connectivity
+
+Your durable function apps connect to the scheduler resource via a gRPC connection. The endpoint address is in the form `{scheduler-name}.{region}.durabletask.io`. For example, `myscheduler-123.westus2.durabletask.io`. The connection is secured using TLS and the app's identity is used to authenticate the connection.
+
+Work items are streamed from the scheduler to the app using a push model, removing the need for polling and improving end-to-end latency. Your apps can process multiple work items in parallel and will send responses back to the scheduler when the corresponding orchestration, activity, or entity task is complete.
 
-### Managed by Azure 
+### State management
 
-Unlike other [existing storage providers](../durable-functions-storage-providers.md) for durable functions, the durable task scheduler offers dedicated resources that are fully managed by Azure. You no longer need to bring your own storage account for storing orchestration and entity state, as it is built in.
+The scheduler manages the state of orchestrations and entities internally. It doesn't require you to provide a separate storage account for state management. The internal state store is highly optimized for use with durable functions, allowing for better performance and reduced latency compared to BYO storage providers.
 
-Since Azure has direct access to your app's backend, it also becomes easier to diagnose and resolve problems related to durable task scheduler so users can expect a better support experience. 
+The scheduler internally uses a combination of in-memory and persistent storage to manage state. The in-memory store is used for short-lived state, while the persistent store is used for recovery and for multi-instance query operations. This bespoke design, heavily inspired by the [Netherite storage architecture](https://www.vldb.org/pvldb/vol15/p1591-burckhardt.pdf), allows the scheduler to provide low-latency access to state while still ensuring durability and reliability.
+
+## Feature highlights
+
+> [!NOTE]
+> The following features are available at the current stage of public preview with more under way. To report problems or request new features, submit an issue in the [durable task scheduler GitHub repository](https://aka.ms/dts-samples).
 
 ### Durable task scheduler dashboard
 
-When a scheduler resource is created, a corresponding dashboard is provided out-of-the-box. The dashboard provides an overview of all orchestrations instances and allows you to quickly filter by different criteria. You can easily gather data about an orchestration instance, such as status, duration, input/output, etc. You can also drill into an instance to get data about sub-orchestrations and activities. The dashboard displays data related to Durable Entities as well. 
+When a scheduler resource is created, a corresponding dashboard is provided out-of-the-box. The dashboard provides an overview of all orchestrations and entity instances and allows you to quickly filter by different criteria. You can easily gather data about an orchestration instance, such as status, duration, input/output, etc. You can also drill into an instance to get data about sub-orchestrations and activities.
 
-Aside from monitoring purposes, you can also perform ad hoc management operations on the dashboard, such as pausing, terminating, or restarting an orchestration instance. See [Debug and manage orchestrations using the durable task scheduler dashboard](./durable-task-scheduler-dashboard.md) for more details. 
+Aside from monitoring, you can also perform management operations on the dashboard, such as pausing, terminating, or restarting an orchestration instance. For more information about the dashboard, see [Debug and manage orchestrations using the durable task scheduler dashboard](./durable-task-scheduler-dashboard.md).
 
-[Access to the dashboard](./develop-with-durable-task-scheduler.md#accessing-dts-dashboard) is secured by identity and role-based access. 
+[Access to the dashboard](./develop-with-durable-task-scheduler.md#accessing-dts-dashboard) is secured by identity and role-based access controls. 
 
-### Supports the highest throughput
+### Multiple task hubs
 
-The durable task scheduler was benchmarked against other storage providers, including the Azure Storage, MSSQL, and Netherite providers. The results show the durable task scheduler provides better throughput than the other options.
+Durable function state is durably persisted in a *task hub*. A [task hub](../durable-functions-task-hubs.md) is a logical container for orchestration and entity instances and provides a way to partition the state store. One scheduler instance allows for the creation of multiple task hubs, each of which can be used by different apps. Access to a task hub requires that the caller's identity has the required role-based access control (RBAC) permissions.
 
-The following table shows the results of a series of benchmarks ran to compare the relative throughput of the durable task scheduler provider vs. the default Azure Storage provider. The Azure Storage provider was chosen as the comparison because it's by far the most commonly used backend option for durable function apps.
+Creating multiple task hubs allows you to isolate different workloads and manage them independently. For example, you can create a task hub for each environment (dev, test, prod) or for different teams within your organization. Creating multiple task hubs in a single scheduler instance is also a way to reduce costs, as you can share the same scheduler resources across multiple task hubs. However, task hubs under the same scheduler instance share the same resources, so if one task hub is heavily loaded, it might affect the performance of the other task hubs.
+
+### Emulator for local development
+
+The [durable task scheduler emulator](./quickstart-durable-task-scheduler.md#set-up-dts-emulator) is a lightweight version of the scheduler backend that runs locally in a Docker container. It allows you to develop and test your durable function app without needing to deploy it to Azure. The emulator provides a local version of the management dashboard, so you can monitor and manage your orchestrations and entities just like you would in Azure.
+
+By default, the emulator exposes a single task hub named `default`. You can expose multiple task hubs by specifying the `DTS_TASK_HUB_NAMES` environment variable with a comma-separated list of task hub names when starting the emulator. For example, to enable two task hubs named `taskhub1` and `taskhub2`, you can run the following command:
+
+```bash
+docker run -d -p 8080:8080 -e DTS_TASK_HUB_NAMES=taskhub1,taskhub2 mcr.microsoft.com/dts/dts-emulator:latest
+```
+
+> [!NOTE]
+> The emulator internally stores orchestration and entity state in local memory, so it isn't suitable for production use.
+
+## Work item throughput
+
+The durable task scheduler was benchmarked against other storage providers, including the Azure Storage, MSSQL, and Netherite providers. The results show the durable task scheduler provides better work item throughput than the other options, which translates into more orchestrator, entity, and activity tasks being processed in a given time period.
+
+The following table shows the results of a series of benchmarks ran to compare the relative throughput of the durable task scheduler provider vs. the default Azure Storage provider. The Azure Storage provider was chosen as the comparison because it's currently the default and most commonly used backend option for durable function apps.
 
 :::image type="content" source="media/durable-task-scheduler/performance.png" alt-text="Bar chart comparing throughput of durable task scheduler vs Azure Storage providers.":::
 
@@ -60,7 +99,7 @@ To test the relative throughput of the backend providers, these benchmarks were
 
 The intent of the benchmark is to measure the overhead of each backend without doing anything too complicated. This type of sequential orchestration was chosen due to its commonality in function apps that include durable functions.
 
-#### Test details
+### Test details
 
 The test consists of the following criteria:  
 
@@ -69,7 +108,7 @@ The test consists of the following criteria:
 - The same app was used for all storage providers, and the only change was the backend storage provider configuration.
 - The test is triggered using an HTTP trigger which starts **5,000 orchestrations concurrently**. 
 
-After the test is triggered, the throughput is calculated by dividing the total number of orchestrations completed by the total time taken to complete them. The test was run multiple times for each storage provider configuration to ensure the results were consistent.
+After the test completes, the throughput is calculated by dividing the total number of completed orchestrations by the total execution time. The test was run multiple times for each storage provider configuration to ensure the results were consistent.
 
 This benchmark showed that the durable task scheduler is roughly **five times faster** than the Azure Storage provider. Your results might vary depending on:
 
@@ -81,35 +120,30 @@ This benchmark showed that the durable task scheduler is roughly **five times fa
 > [!NOTE]
 > These results are meant to provide a rough comparison of the relative performance of the storage provider backends at the time the test was run. These results shouldn't be taken as definitive.
 
-### Supports multiple task hubs
+## Limitations and considerations  
 
-Durable function state is durably persisted in a *task hub*. A [task hub](../durable-functions-task-hubs.md) is a logical container for orchestration and entity instances and provides a way to partition the state store. One scheduler instance allows for the creation of multiple task hubs, each of which can be used by different apps. Access to a task hub requires that the caller's identity has the required RBAC (role-based access control) permissions.
+- **Supported hosting plans**: The durable task scheduler currently only supports durable functions running on *Functions Premium* and *App Service* plans. For apps running on the Functions Premium plan, you must [enable the *Runtime Scale Monitoring* setting](./develop-with-durable-task-scheduler.md#auto-scaling-in-functions-premium-plan) to get auto scaling of the app.
 
-Creating multiple task hubs allows you to isolate different workloads and manage them independently. For example, you can create a task hub for each environment (dev, test, prod) or for different teams within your organization. Creating multiple task hubs in a single scheduler instance is also a way to reduce costs, as you can share the same scheduler resources across multiple task hubs. However, task hubs under the same scheduler instance share the same resources, so if one task hub is heavily loaded, it may affect the performance of the other task hubs.
+  The *Consumption*, *Flex Consumption*, and *Azure Container App* hosting plans aren't yet supported when using the durable task scheduler.
 
-### Emulator for local development
-
-The [durable task scheduler emulator](./quickstart-durable-task-scheduler.md#set-up-dts-emulator) is a lightweight version of the scheduler backend that runs locally in a Docker container. It allows you to develop and test your durable function app without needing to deploy it to Azure. The emulator provides a local version of the management dashboard, so you can monitor and manage your orchestrations and entities just like you would in Azure.
+- **Available regions**: Durable task scheduler resources can be created in a subset of Azure regions today. You can run the following command to get a list of the supported regions:  
 
-Without any additional configuration, the emulator exposes a task hub named `default`. You can enable additional task hubs by specifying the `DTS_TASK_HUB_NAMES` environment variable with a comma-separated list of task hub names when starting the emulator. For example, to enable two task hubs named `taskhub1` and `taskhub2`, you can run the following command:
+    ```bash
+    az provider show --namespace Microsoft.DurableTask --query "resourceTypes[?resourceType=='schedulers'].locations | [0]" --out table
+    ```
 
-```bash
-docker run -d -p 8080:8080 -e DTS_TASK_HUB_NAMES=taskhub1,taskhub2 mcr.microsoft.com/dts/dts-emulator:latest
-```
+    Consider using the same region for your durable functions app and the durable task scheduler resources. Having these resources in different regions might impact performance and limit certain network-related functionality.
 
-> [!NOTE]
-> The emulator internally stores orchestration and entity state in local memory, so it isn't suitable for production use.
+- **Scheduler quota**: You can currently create up to **five schedulers per region** per subscription. 
 
-## Considerations  
+- **Feature parity**: Some features might not be available in the durable task scheduler backend yet. For example, at the time of writing, the durable task scheduler doesn't support the following features:
 
-- **Scaling**: If your durable functions app is not reaching the maximum throughout of the durable task scheduler, consider *scaling out* your app to more instances as the bottleneck might lie in the number of workers available to run orchestrations. If scaling out also doesn't achieve the throughput you need, then consider *scaling up* your compute resources. In the future, you'll also be able to scale out the resources allocated to a durable task scheduler by purchasing more [capacity units](./dts-dedicated-sku.md#dedicated-sku-concepts). 
-- **Supported hosting plans**: The durable task scheduler currently only supports durable functions running on *Functions Premium* and *App Service* plans. For apps running on the Functions Premium plan, [enable the *Runtime Scale Monitoring* setting](./develop-with-durable-task-scheduler.md#auto-scaling-in-functions-premium-plan) to get auto scaling of the app.
-- **Available regions**: Durable task scheduler is only available in certain Azure regions today. Run this command to get the latest supported regions:  
+    - [Orchestration rewind](../durable-functions-instance-management#rewind-instances-preview)
+    - [Extended sessions](../durable-functions-azure-storage-provider#extended-sessions)
+    - [Management operations using the Azure Functions Core Tools](../durable-functions-instance-management#azure-functions-core-tools)
 
-    `az provider show --namespace Microsoft.DurableTask --query "resourceTypes[?resourceType=='schedulers'].locations | [0]" --out table`. 
-    
-    Consider picking the same region for your durable functions app and the durable task scheduler as having these resources in different regions may impact performance and limit certain network-related functionality.
-- **Scheduler quota**: You can create up to **five schedulers per region** per subscription today. 
+    > [!NOTE]
+    > Feature availability is subject to change as the durable task scheduler backend approaches general availability.
 
 ## Next steps