Merge pull request #227545 from lilyjma/DFBestPractice

Durable Functions best practice reference

Author: denrea

articles/azure-functions/durable/TOC.yml

@@ -136,6 +136,10 @@
   items:
   - name: Develop
     items:
+    - name: Durable Functions best practices and diagnostic tools
+      href: durable-functions-best-practice-reference.md
+    - name: Durable Functions Rosyln Analyzer (C# only)
+      href: durable-functions-roslyn-analyzer.md
     - name: Create in Azure portal
       href: durable-functions-create-portal.md
     - name: Manage connections
@@ -162,8 +166,14 @@
       href: durable-functions-diagnostics.md
     - name: Monitoring
       href: ../functions-monitoring.md?toc=/azure/azure-functions/durable/toc.json
+    - name: App diagnostics in Azure portal 
+      href: function-app-diagnostics.md
     - name: Pricing
       href: durable-functions-billing.md
+  - name: Debug
+    items:
+    - name: Upgrade Durable Functions extension version
+      href: durable-functions-extension-upgrade.md
 - name: Resources
   items:
   - name: Build your skills with Microsoft Learn training

articles/azure-functions/durable/durable-functions-best-practice-reference.md

@@ -0,0 +1,106 @@
+---
+title: Durable Functions best practices and diagnostic tools
+description: Learn about the best practices when using Durable Functions and the various tools available for diagnosing problems.
+author: lilyjma
+ms.topic: conceptual
+ms.date: 02/15/2023
+ms.author: azfuncdf
+---
+
+# Durable Functions best practices and diagnostic tools
+
+This article details some best practices when using Durable Functions. It also describes various tools to help diagnose problems during development, testing, and production use.
+
+## Best practices
+
+### Use the latest version of the Durable Functions extension and SDK
+
+There are two components that a function app uses to execute Durable Functions. One is the *Durable Functions SDK* that allows you to write orchestrator, activity, and entity functions using your target programming language. The other is the *Durable extension*, which is the runtime component that actually executes the code. With the exception of .NET in-process apps, the SDK and the extension are versioned independently.
+    
+Staying up to date with the latest extension and SDK ensures your application benefits from the latest performance improvements, features, and bug fixes. Upgrading to the latest versions also ensures that Microsoft can collect the latest diagnostic telemetry to help accelerate the investigation process when you open a support case with Azure.
+    
+* See [Upgrade durable functions extension version](durable-functions-extension-upgrade.md) for instructions on getting the latest extension version.
+* To ensure you're using the latest version of the SDK, check the package manager of the language you're using. 
+
+### Adhere to Durable Functions [code constraints](durable-functions-code-constraints.md)
+
+The [replay](durable-functions-orchestrations.md#reliability) behavior of orchestrator code creates constraints on the type of code that you can write in an orchestrator function. An example of a constraint is that your orchestrator function must use deterministic APIs so that each time it’s replayed, it produces the same result.
+
+> [!NOTE]
+> The Durable Functions Roslyn Analyzer is a live code analyzer that guides C# users to adhere to Durable Functions specific code constraints. See [Durable Functions Roslyn Analyzer](durable-functions-roslyn-analyzer.md) for instructions on how to enable it on Visual Studio and Visual Studio Code.  
+
+### Familiarize yourself with your programming language's Azure Functions performance settings
+
+_Using default settings_, the language runtime you select may impose strict concurrency restrictions on your functions. For example: only allowing 1 function to execute at a time on a given VM. These restrictions can usually be relaxed by _fine tuning_ the concurrency and performance settings of your language. If you're looking to optimize the performance of your Durable Functions application, you will need to familiarize yourself with these settings.
+
+Below is a non-exhaustive list of some of the languages that often benefit from fine tuning their performance and concurrency settings, and their guidelines for doing so.
+
+* [JavaScript](../functions-reference-node.md#scaling-and-concurrency)
+* [PowerShell](../functions-reference-powershell.md#concurrency)
+* [Python](../python-scale-performance-reference.md)
+
+### Guarantee unique Task Hub names per app
+
+Multiple Durable Function apps can share the same storage account. By default, the name of the app is used as the task hub name, which ensures that accidental sharing of task hubs won't happen. If you need to explicitly configure task hub names for your apps in host.json, you must ensure that the names are [*unique*](durable-functions-task-hubs.md#multiple-function-apps). Otherwise, the multiple apps will compete for messages, which could result in undefined behavior, including orchestrations getting unexpectedly "stuck" in the Pending or Running state. 
+
+The only exception is if you deploy *copies* of the same app in [multiple regions](durable-functions-disaster-recovery-geo-distribution.md); in this case, you can use the same task hub for the copies. 
+
+### Follow guidance when deploying code changes to running orchestrators
+
+It's inevitable that functions will be added, removed, and changed over the lifetime of an application. Examples of [common breaking changes](durable-functions-versioning.md) include changing activity or entity function signatures and changing orchestrator logic. These changes are a problem when they affect orchestrations that are still running. If deployed incorrectly, code changes could lead to orchestrations failing with a non-deterministic error, getting stuck indefinitely, performance degradation, etc. Refer to recommended [mitigation strategies](durable-functions-versioning.md#mitigation-strategies) when making code changes that may impact running orchestrations. 
+
+### Keep function inputs and outputs as small as possible
+
+You can run into memory issues if you provide large inputs and outputs to and from Durable Functions APIs. 
+
+Inputs and outputs to Durable Functions APIs are serialized into the orchestration history. This means that large inputs and outputs can, over time, greatly contribute to an orchestrator history growing unbounded, which risks causing memory exceptions during [replay](durable-functions-orchestrations.md#reliability).
+
+To mitigate the impact of large inputs and outputs to APIs, you may choose to delegate some work to sub-orchestrators. This helps load balance the history memory burden from a single orchestrator to multiple ones, therefore keeping the memory footprint of individual histories small.
+
+That said the best practice for dealing with _large_ data is to keep it in external storage and to only materialize that data inside Activities, when needed. When taking this approach, instead of communicating the data itself as inputs and/or outputs of Durable Functions APIs, you can pass in some lightweight identifier that allows you to retrieve that data from external storage when needed in your Activities.
+
+### Fine tune your Durable Functions concurrency settings
+
+A single worker instance can execute multiple work items concurrently to increase efficiency. However, processing too many work items concurrently risks exhausting resources like CPU capacity, network connections, etc. In many cases, this shouldn’t be a concern because scaling and limiting work items are handled automatically for you. That said, if you’re experiencing performance issues (such as orchestrators taking too long to finish, are stuck in pending, etc.) or are doing performance testing, you could [configure concurrency limits](durable-functions-perf-and-scale.md#configuration-of-throttles) in the host.json file.
+
+> [!NOTE]
+> This is not a replacement for fine-tuning the performance and concurrency settings of your language runtime in Azure Functions. The Durable Functions concurrency settings only determine how much work can be assigned to a given VM at a time, but it does not determine the degree of parallelism in processing that work inside the VM. The latter requires fine-tuning the language runtime performance settings.
+ 
+ 
+## Diagnostic tools
+
+There are several tools available to help you diagnose problems.
+
+### Durable Functions and Durable Task Framework Logs
+
+#### Durable Functions Extension
+The Durable extension emits tracking events that allow you to trace the end-to-end execution of an orchestration. These tracking events can be found and queried using the [Application Insights Analytics](../../azure-monitor/logs/log-query-overview.md) tool in the Azure portal. The verbosity of tracking data emitted can be configured in the `logger` (Functions 1.x) or `logging` (Functions 2.0) section of the host.json file. See [configuration details](durable-functions-diagnostics.md#functions-10). 
+        
+#### Durable Task Framework
+Starting in v2.3.0 of the Durable extension, logs emitted by the underlying Durable Task Framework (DTFx) are also available for collection. See [details on how to enable these logs](durable-functions-diagnostics.md#durable-task-framework-logging).  
+
+### Azure portal
+
+#### Diagnose and solve problems
+Azure Function App Diagnostics is a useful resource on Azure portal for monitoring and diagnosing potential issues in your application. It also provides suggestions to help resolve problems based on the diagnosis. See [Azure Function App Diagnostics](function-app-diagnostics.md). 
+
+#### Durable Functions Orchestration traces
+Azure portal provides orchestration trace details to help you understand the status of each orchestration instance and trace the end-to-end execution. When you look at the list of functions inside your Azure Functions app, you'll see a **Monitor** column that contains links to the traces. You need to have Applications Insights enabled for your app to get this information. 
+
+### Durable Functions Monitor Extension
+
+This is a [Visual Studio Code extension](https://github.com/microsoft/DurableFunctionsMonitor) that provides a UI for monitoring, managing, and debugging your orchestration instances. 
+
+### Roslyn Analyzer
+
+The Durable Functions Roslyn Analyzer is a live code analyzer that guides C# users to adhere to Durable Functions specific [code constraints](durable-functions-code-constraints.md). See [Durable Functions Roslyn Analyzer](durable-functions-roslyn-analyzer.md) for instructions on how to enable it on Visual Studio and Visual Studio Code. 
+
+
+## Support
+
+For questions and support, you may open an issue in one of the GitHub repos below. When reporting a bug in Azure, including information such as affected instance IDs, time ranges in UTC showing the problem, the application name (if possible) and deployment region will greatly speed up investigations.
+- [Durable Functions extension and .NET in-process SDK](https://github.com/Azure/azure-functions-durable-extension/issues)
+- [.NET isolated SDK](https://github.com/microsoft/durabletask-dotnet/issues)
+- [Durable Functions for Java](https://github.com/microsoft/durabletask-java/issues)
+- [Durable Functions for JavaScript](https://github.com/Azure/azure-functions-durable-js/issues)
+- [Durable Functions for Python](https://github.com/Azure/azure-functions-durable-python/issues)

articles/azure-functions/durable/durable-functions-extension-upgrade.md

@@ -0,0 +1,66 @@
+---
+title: Upgrade Durable Functions extension version
+description: Learn why it's important to use the latest version of the Durable Functions extension and how to upgrade to the latest.
+author: lilyjma
+ms.topic: conceptual
+ms.date: 02/15/2023
+ms.author: azfuncdf
+---
+
+# Upgrade Durable Functions extension version
+
+
+Many issues users experience with Durable Functions can be resolved simply by upgrading to the latest version of the extension, which often contains important bug fixes and performance improvements. You can follow the instructions in this article to get the latest version of the Durable Functions extension. 
+
+Changes to the extension can be found in the [Release page](https://github.com/Azure/azure-functions-durable-extension/releases) of the `Azure/azure-functions-durable-extension` repo. You can also configure to receive notifications whenever there's a new extension release by going to the **Releases page**, clicking on **Watch**, then on **Custom**, and finally selecting the **Releases** filter:
+
+:::image type="content" source="media/durable-functions-best-practice/watch-releases-1.png" alt-text="Screenshot of step 1 to set up release notifications.":::
+
+:::image type="content" source="media/durable-functions-best-practice/watch-releases-2.png" alt-text="Screenshot of step 2 to set up release notifications.":::
+
+## Reference the latest NuGet packages (.NET apps only)
+.NET apps can get the latest version of the Durable Functions extension by referencing the latest NuGet package: 
+
+* [.NET in-process worker](https://www.nuget.org/packages/Microsoft.Azure.WebJobs.Extensions.DurableTask)
+* [.NET isolated worker](https://www.nuget.org/packages/Microsoft.Azure.Functions.Worker.Extensions.DurableTask)
+
+If you're using the Netherite or MSSQL [storage providers](durable-functions-storage-providers.md) (instead of Azure Storage), you need to reference one of the following:
+
+* [Netherite, in-process worker](https://www.nuget.org/packages/Microsoft.Azure.DurableTask.Netherite.AzureFunctions)
+* [Netherite, isolated worker](https://www.nuget.org/packages/Microsoft.Azure.Functions.Worker.Extensions.DurableTask.Netherite)
+* [MSSQL, in-process worker](https://www.nuget.org/packages/Microsoft.DurableTask.SqlServer.AzureFunctions)
+* [MSSQL, isolated worker](https://www.nuget.org/packages/Microsoft.Azure.Functions.Worker.Extensions.DurableTask.SqlServer)
+
+## Upgrade the extension bundle 
+[Extension bundles](../functions-bindings-register.md#extension-bundles) provide an easy and convenient way for non-.NET function apps to reference and use various Azure Function triggers and bindings. For example, if you need to send a message to Event Hubs every time your function is triggered, you can use the Event Hubs extension to gain access to Event Hubs bindings. The Durable Functions extension is also included in each version of extension bundles. Extension bundles are automatically configured in host.json when creating a function app using any of the supported development tools. 
+
+Most non-.NET applications rely on extension bundles to gain access to various triggers and bindings. The [latest bundle release](https://github.com/Azure/azure-functions-extension-bundles) often contains the latest version of the Durable Functions extension with critical bug fixes and performance improvements. Therefore, it's important that your app uses the latest version of extension bundles. You can check your host.json file to see whether the version range you're using includes the latest extension bundle version. 
+
+## Manually upgrade the Durable Functions extension
+If upgrading the extension bundle didn't resolve your problem, and you noticed a newer release of the Durable Functions extension containing a potential fix to your problem, then you could try to manually upgrade the extension itself. Note that this is only intended for advanced scenarios or when time-sensitive fixes are necessary as there are many drawbacks to manually managing extensions. For example, you may have to deal to .NET errors when the extensions you use are incompatible with each other. You also need to manually upgrade extensions to get the latest fixes and patches instead of getting them automatically through the extension bundle.
+
+First, remove the `extensionBundle` section from your host.json file.
+
+Install the `dotnet` CLI if you don't already have it. You can get it from this [page](https://www.microsoft.com/net/download/).
+
+Because applications normally use more than one extension, it's recommended that you run the following to manually install all the latest version of all extensions supported by Extension Bundles:
+
+```console
+func extensions install
+```
+
+However, if you **only** wish to install the latest Durable Functions extension release, you would run the following command: 
+
+```console
+func extensions install Microsoft.Azure.WebJobs.Extensions.DurableTask -v <version>
+```
+
+For example:
+
+```console
+func extensions install Microsoft.Azure.WebJobs.Extensions.DurableTask -v 2.9.1
+```
+
+
+
+

articles/azure-functions/durable/durable-functions-roslyn-analyzer.md

@@ -0,0 +1,35 @@
+---
+title: Durable Functions Roslyn Analyzer (C# only)
+description: Learn about how to use the Roslyn Analyzer to help adhere to Durable Functions specific code constraints.
+author: amdeel
+ms.topic: conceptual
+ms.date: 02/15/2023
+ms.author: azfuncdf
+---
+
+# Durable Functions Rosyln Analyzer (C# only)
+
+The Durable Functions Roslyn Analyzer is a live code analyzer that guides C# users to adhere to Durable Functions specific [code constraints](./durable-functions-code-constraints.md). This analyzer is enabled by default to check your Durable Functions code and generate warnings and errors when there's any. Currently, the analyzer is only supported in the .NET in-process worker.
+
+For more detailed information on the analyzer (improvements, releases, bug fixes, etc.), see its [release notes page](https://github.com/Azure/azure-functions-durable-extension/releases/tag/Analyzer-v0.2.0).
+
+
+## Configuration
+
+### Visual Studio
+
+For the best experience, you'll want to enable full solution analysis in your Visual Studio settings. This can be done by going to **Tools** -> **Options** -> **Text Editor** -> **C#** -> **Advanced** -> **"Entire solution"**:
+
+:::image type="content" source="media/durable-functions-best-practice/roslyn-analyzer-1.png" alt-text="Screenshot of configuring Roslyn Analyzer in Visual Studio.":::
+
+Depending on the version of Visual Studio, you may also see "Enable full solution analysis": 
+
+:::image type="content" source="media/durable-functions-best-practice/roslyn-analyzer-2.png" alt-text="Screenshot of configuring Roslyn Analyzer in another version of Visual Studio.":::
+
+To disable the analyzer, refer to these [instructions](/visualstudio/code-quality/in-source-suppression-overview). 
+
+### Visual Studio Code
+
+Open **Settings** by clicking the wheel icon on the lower left corner, then search for “rosyln”. “Enable Rosyln Analyzers” should show up as one of the results. Check the enable support box.
+
+:::image type="content" source="media/durable-functions-best-practice/roslyn-analyzer-vs-code.png" alt-text="Screenshot of configuring Roslyn Analyzer in Visual Studio Code.":::