From 33332bd8390e1baf906c844a1ede7a9e2ff4b14e Mon Sep 17 00:00:00 2001 From: Genevieve Warren <24882762+gewarren@users.noreply.github.com> Date: Fri, 25 Jul 2025 08:29:37 -0700 Subject: [PATCH 01/10] Add new evaluators and package (#47569) --- docs/ai/conceptual/evaluation-libraries.md | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/docs/ai/conceptual/evaluation-libraries.md b/docs/ai/conceptual/evaluation-libraries.md index 9fb92323e0833..377248b45fa97 100644 --- a/docs/ai/conceptual/evaluation-libraries.md +++ b/docs/ai/conceptual/evaluation-libraries.md @@ -2,7 +2,7 @@ title: The Microsoft.Extensions.AI.Evaluation libraries description: Learn about the Microsoft.Extensions.AI.Evaluation libraries, which simplify the process of evaluating the quality and accuracy of responses generated by AI models in .NET intelligent apps. ms.topic: concept-article -ms.date: 05/13/2025 +ms.date: 07/24/2025 --- # The Microsoft.Extensions.AI.Evaluation libraries @@ -11,8 +11,9 @@ The Microsoft.Extensions.AI.Evaluation libraries simplify the process of evaluat The evaluation libraries, which are built on top of the [Microsoft.Extensions.AI abstractions](../microsoft-extensions-ai.md), are composed of the following NuGet packages: - [📦 Microsoft.Extensions.AI.Evaluation](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation) – Defines the core abstractions and types for supporting evaluation. -- [📦 Microsoft.Extensions.AI.Evaluation.Quality](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Quality) – Contains evaluators that assess the quality of LLM responses in an app according to metrics such as relevance and completeness. These evaluators use the LLM directly to perform evaluations. -- [📦 Microsoft.Extensions.AI.Evaluation.Safety](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Safety) – Contains evaluators, such as the `ProtectedMaterialEvaluator` and `ContentHarmEvaluator`, that use the [Azure AI Foundry](/azure/ai-foundry/) Evaluation service to perform evaluations. +- [📦 Microsoft.Extensions.AI.Evaluation.NLP](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.NLP) - Contains [evaluators](#nlp-evaluators) that evaluate the similarity of an LLM's response text to one or more reference responses using natural language processing (NLP) metrics. These evaluators aren't LLM or AI-based; they use traditional NLP techniques such as text tokenization and n-gram analysis to evaluate text similarity. +- [📦 Microsoft.Extensions.AI.Evaluation.Quality](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Quality) – Contains [evaluators](#quality-evaluators) that assess the quality of LLM responses in an app according to metrics such as relevance and completeness. These evaluators use the LLM directly to perform evaluations. +- [📦 Microsoft.Extensions.AI.Evaluation.Safety](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Safety) – Contains [evaluators](#safety-evaluators), such as the `ProtectedMaterialEvaluator` and `ContentHarmEvaluator`, that use the [Azure AI Foundry](/azure/ai-foundry/) Evaluation service to perform evaluations. - [📦 Microsoft.Extensions.AI.Evaluation.Reporting](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Reporting) – Contains support for caching LLM responses, storing the results of evaluations, and generating reports from that data. - [📦 Microsoft.Extensions.AI.Evaluation.Reporting.Azure](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Reporting.Azure) - Supports the reporting library with an implementation for caching LLM responses and storing the evaluation results in an [Azure Storage](/azure/storage/common/storage-introduction) container. - [📦 Microsoft.Extensions.AI.Evaluation.Console](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Console) – A command-line tool for generating reports and managing evaluation data. @@ -23,7 +24,7 @@ The libraries are designed to integrate smoothly with existing .NET apps, allowi ## Comprehensive evaluation metrics -The evaluation libraries were built in collaboration with data science researchers from Microsoft and GitHub, and were tested on popular Microsoft Copilot experiences. The following sections show the built-in [quality](#quality-evaluators) and [safety](#safety-evaluators) evaluators and the metrics they measure. +The evaluation libraries were built in collaboration with data science researchers from Microsoft and GitHub, and were tested on popular Microsoft Copilot experiences. The following sections show the built-in [quality](#quality-evaluators), [NLP](#nlp-evaluators), and [safety](#safety-evaluators) evaluators and the metrics they measure. You can also customize to add your own evaluations by implementing the interface. @@ -41,9 +42,22 @@ Quality evaluators measure response quality. They use an LLM to perform the eval | | `Equivalence` | Evaluates the similarity between the generated text and its ground truth with respect to a query | | | `Groundedness` | Evaluates how well a generated response aligns with the given context | | † | `Relevance (RTC)`, `Truth (RTC)`, and `Completeness (RTC)` | Evaluates how relevant, truthful, and complete a response is | +| | `Intent Resolution` | Evaluates an AI system's effectiveness at identifying and resolving user intent (agent-focused) | +| | `Task Adherence` | Evaluates an AI system's effectiveness at adhering to the task assigned to it (agent-focused) | +| | `Tool Call Accuracy` | Evaluates an AI system's effectiveness at using the tools supplied to it (agent-focused) | † This evaluator is marked [experimental](../../fundamentals/syslib-diagnostics/experimental-overview.md). +### NLP evaluators + +NLP evaluators evaluate the quality of an LLM response by comparing it to a reference response using natural language processing (NLP) techniques. These evaluators aren't LLM or AI-based; instead, they use older NLP techniques to perform text comparisons. + +| Evaluator type | Metric | Description | +|---------------------------------------------------------------------------|--------------------|-------------| +| | `BLEU` | Evaluates a response by comparing it to one or more reference responses using the bilingual evaluation understudy (BLEU) algorithm. This algorithm is commonly used to evaluate the quality of machine-translation or text-generation tasks. | +| | `GLEU` | Measures the similarity between the generated response and one or more reference responses using the Google BLEU (GLEU) algorithm, a variant of the BLEU algorithm that's optimized for sentence-level evaluation. | +| | `F1` | Evaluates a response by comparing it to a reference response using the *F1* scoring algorithm (the ratio of the number of shared words between the generated response and the reference response). | + ### Safety evaluators Safety evaluators check for presence of harmful, inappropriate, or unsafe content in a response. They rely on the Azure AI Foundry Evaluation service, which uses a model that's fine tuned to perform evaluations. From 10ed458bc7fe47b166f0ce5ae0fce44b05ad053e Mon Sep 17 00:00:00 2001 From: Bartosz Klonowski <70535775+BartoszKlonowski@users.noreply.github.com> Date: Fri, 25 Jul 2025 19:54:14 +0200 Subject: [PATCH 02/10] Correct the typo of list instead of dry-run (#47098) --- docs/core/additional-tools/uninstall-tool-cli-dry-run.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/core/additional-tools/uninstall-tool-cli-dry-run.md b/docs/core/additional-tools/uninstall-tool-cli-dry-run.md index 83224da560b04..6fc6fb56a85d8 100644 --- a/docs/core/additional-tools/uninstall-tool-cli-dry-run.md +++ b/docs/core/additional-tools/uninstall-tool-cli-dry-run.md @@ -49,7 +49,7 @@ dotnet-core-uninstall dry-run -h|--help|-? ## Description -The `dotnet-core-uninstall list` command simulates .NET SDK and runtime removal. A status output is provided for each .NET SDK and runtime that would have been removed by the tool. +The `dotnet-core-uninstall dry-run` command simulates .NET SDK and runtime removal. A status output is provided for each .NET SDK and runtime that would have been removed by the tool. ### Arguments From ac4f0e0cd6fcecc083d665d6b549eab9d7115ca4 Mon Sep 17 00:00:00 2001 From: Damian Date: Fri, 25 Jul 2025 19:11:02 +0100 Subject: [PATCH 03/10] Update sentiment-analysis-model-builder.md (#46584) Correcting an incorrect extension used in the documentation related to a generated ML.NET model file. --- .../tutorials/sentiment-analysis-model-builder.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/machine-learning/tutorials/sentiment-analysis-model-builder.md b/docs/machine-learning/tutorials/sentiment-analysis-model-builder.md index 074f198afd62a..b2edf6ca562a2 100644 --- a/docs/machine-learning/tutorials/sentiment-analysis-model-builder.md +++ b/docs/machine-learning/tutorials/sentiment-analysis-model-builder.md @@ -98,7 +98,7 @@ The machine learning task used to train the sentiment analysis model in this tut - *SentimentAnalysis.consumption.cs* - This file contains the `ModelInput` and `ModelOutput` schemas as well as the `Predict` function generated for consuming the model. - *SentimentAnalysis.training.cs* - This file contains the training pipeline (data transforms, trainer, trainer hyperparameters) chosen by Model Builder to train the model. You can use this pipeline for re-training your model. - - **SentimentAnalysis.zip* - This is a serialized zip file which represents your trained ML.NET model. + - **SentimentAnalysis.mlnet* - This file contains metadata and configuration details for an ML.NET model. 1. Select the **Next step** button to move to the next step. @@ -144,7 +144,7 @@ To make a single prediction, you have to create a () - .FromFile("SentimentAnalysis.zip"); + .FromFile("SentimentAnalysis.mlnet"); ``` ### Create sentiment analysis handler From ac0540db5cec4a288b94100c9b3ece6aa188f8e4 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Fri, 25 Jul 2025 13:37:58 -0500 Subject: [PATCH 04/10] Document breaking change: GnuTarEntry and PaxTarEntry no longer includes atime and ctime by default (#46588) --- docs/core/compatibility/10.0.md | 1 + .../10.0/tar-atime-ctime-default.md | 57 +++++++++++++++++++ docs/core/compatibility/toc.yml | 2 + 3 files changed, 60 insertions(+) create mode 100644 docs/core/compatibility/core-libraries/10.0/tar-atime-ctime-default.md diff --git a/docs/core/compatibility/10.0.md b/docs/core/compatibility/10.0.md index 30ebe4963f6b3..1b71ea787cc16 100644 --- a/docs/core/compatibility/10.0.md +++ b/docs/core/compatibility/10.0.md @@ -30,6 +30,7 @@ If you're migrating an app to .NET 10, the breaking changes listed here might af | [Consistent shift behavior in generic math](core-libraries/10.0/generic-math.md) | Behavioral change | Preview 1 | | [Default trace context propagator updated to W3C standard](core-libraries/10.0/default-trace-context-propagator.md) | Behavioral change | Preview 4 | | [DriveInfo.DriveFormat returns Linux filesystem types](core-libraries/10.0/driveinfo-driveformat-linux.md) | Behavioral change | Preview 6 | +| [GnuTarEntry and PaxTarEntry no longer includes atime and ctime by default](core-libraries/10.0/tar-atime-ctime-default.md) | Behavioral change | Preview 5 | | [LDAP DirectoryControl parsing is now more stringent](core-libraries/10.0/ldap-directorycontrol-parsing.md) | Behavioral change | Preview 1 | | [MacCatalyst version normalization](core-libraries/10.0/maccatalyst-version-normalization.md) | Behavioral change | Preview 1 | | [.NET runtime no longer provides default SIGTERM signal handler](core-libraries/10.0/sigterm-signal-handler.md) | Behavioral change | Preview 5 | diff --git a/docs/core/compatibility/core-libraries/10.0/tar-atime-ctime-default.md b/docs/core/compatibility/core-libraries/10.0/tar-atime-ctime-default.md new file mode 100644 index 0000000000000..49a023cfa3d9a --- /dev/null +++ b/docs/core/compatibility/core-libraries/10.0/tar-atime-ctime-default.md @@ -0,0 +1,57 @@ +--- +title: "Breaking change: GnuTarEntry and PaxTarEntry exclude atime and ctime by default" +description: "Learn about the breaking change in .NET 10 where GnuTarEntry and PaxTarEntry no longer automatically set access time and change time fields." +ms.date: 06/05/2025 +ai-usage: ai-assisted +ms.custom: https://github.com/dotnet/docs/issues/46465 +--- +# GnuTarEntry and PaxTarEntry exclude atime and ctime by default + +Starting in .NET 10, and no longer automatically set access time (`atime`) and change time (`ctime`) fields when creating new entries. These fields are problematic in tar entries because not all tar readers support them. The fields will still be preserved when reading, and you set them directly. But they won't be set on existing entries that didn't have them to start, or when converting from other entry types. + +The behavior of is unchanged. It's initialized to for tar entries created with a constructor, and uses the file modification time for entries created from files. + +Other minor fixes have been made to to prioritize round-tripping of `TarEntry` entries from read to write without change. + +## Version introduced + +.NET 10 Preview 5 + +## Previous behavior + +Previously, `GnuTarEntry` and `PaxTarEntry` always added `atime` and `ctime` values when creating new entries. + +## New behavior + +Starting in .NET 10, `GnuTarEntry` and `PaxTarEntry` only set `atime` and `ctime` if: + +- The entry is read from a tar archive that already contains these fields. +- The user explicitly set them using the appropriate properties. + +The behavior of remains unchanged. It's initialized to for tar entries created with a constructor, and uses the file modification time for entries created from files. + +## Type of breaking change + +This is a [behavioral change](../../categories.md#behavioral-change). + +## Reason for change + +This change was made for better compatibility with other tar readers and to improve round-tripping of tar files without modification. + +## Recommended action + +No action required for most users—most archives have no use for these timestamps. + +If you require these fields to be set, you can use the following APIs: + +- For `GnuTarEntry`: Use the and properties. +- For `PaxTarEntry`: Use the constructor that accepts extended attributes: . + +However, be aware that setting these fields creates a tar file that isn't readable by many tar clients. + +## Affected APIs + +- +- +- +- diff --git a/docs/core/compatibility/toc.yml b/docs/core/compatibility/toc.yml index a61f94b5fb29e..bef069b8a508c 100644 --- a/docs/core/compatibility/toc.yml +++ b/docs/core/compatibility/toc.yml @@ -26,6 +26,8 @@ items: href: core-libraries/10.0/default-trace-context-propagator.md - name: DriveInfo.DriveFormat returns Linux filesystem types href: core-libraries/10.0/driveinfo-driveformat-linux.md + - name: GnuTarEntry and PaxTarEntry exclude atime and ctime by default + href: core-libraries/10.0/tar-atime-ctime-default.md - name: LDAP DirectoryControl parsing is now more stringent href: core-libraries/10.0/ldap-directorycontrol-parsing.md - name: MacCatalyst version normalization From 5f3514c440e5528b165f91781edd6dea25ea8519 Mon Sep 17 00:00:00 2001 From: Genevieve Warren <24882762+gewarren@users.noreply.github.com> Date: Fri, 25 Jul 2025 11:39:07 -0700 Subject: [PATCH 05/10] Clarify it doesn't just affect default value (#47513) --- docs/core/compatibility/sdk/8.0/source-link.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/core/compatibility/sdk/8.0/source-link.md b/docs/core/compatibility/sdk/8.0/source-link.md index ca4e7894299d9..0e9cd8824743e 100644 --- a/docs/core/compatibility/sdk/8.0/source-link.md +++ b/docs/core/compatibility/sdk/8.0/source-link.md @@ -12,11 +12,11 @@ The [Source Link](https://github.com/dotnet/sourcelink) build tooling is now inc ## Previous behavior -Prior to this change, the default `InformationalVersion` of a library or application was the `Version` property. +Prior to this change, `InformationalVersion` did not include any build metadata. The default `InformationalVersion` of a library or application was the `Version` property. ## New behavior -Starting in .NET 8, the default `InformationalVersion` of a library or application is the `Version` property *and* the `SourceRevisionId` property. +Starting in .NET 8, `InformationalVersion` includes the `SourceRevisionId` property in all cases. For example, the default `InformationalVersion` of a library or application is the `Version` property *and* the `SourceRevisionId` property. ## Version introduced From 6749214a23b0bf6ddcbcc0a8d72625140d1f29b5 Mon Sep 17 00:00:00 2001 From: Genevieve Warren <24882762+gewarren@users.noreply.github.com> Date: Fri, 25 Jul 2025 11:39:40 -0700 Subject: [PATCH 06/10] Resolved-by-customer improvement (#47524) --- .github/policies/label-issues.yml | 20 ++++++++------------ 1 file changed, 8 insertions(+), 12 deletions(-) diff --git a/.github/policies/label-issues.yml b/.github/policies/label-issues.yml index af3fd34ba4780..c4ef0ed4d55cf 100644 --- a/.github/policies/label-issues.yml +++ b/.github/policies/label-issues.yml @@ -31,14 +31,12 @@ configuration: * 'needs-more-info' label removed if: - payloadType: Issues - - or: - + - or: - or: - isAction: action: Opened - isAction: - action: Reopened - + action: Reopened - labelRemoved: label: 'needs-more-info' then: @@ -57,8 +55,7 @@ configuration: - isAction: action: Closed - isActivitySender: - issueAuthor: true - + issueAuthor: true - labelAdded: label: ':world_map: reQUEST' then: @@ -71,7 +68,6 @@ configuration: - or: - payloadType: Issues - payloadType: Pull_Request - - labelAdded: label: ':world_map: mapQUEST' then: @@ -97,8 +93,7 @@ configuration: - isAction: action: Reopened - hasLabel: - label: "won't fix" - + label: "won't fix" then: - removeLabel: label: "won't fix" @@ -109,8 +104,7 @@ configuration: - isAction: action: Closed - hasLabel: - label: 'in-progress' - + label: 'in-progress' then: - removeLabel: label: 'in-progress' @@ -122,7 +116,9 @@ configuration: action: Closed - isActivitySender: issueAuthor: true - + - not: + activitySenderHasAssociation: + association: Member then: - addLabel: label: resolved-by-customer From b18832eba78c7de2b7b6e317755eedb876df0ee2 Mon Sep 17 00:00:00 2001 From: Genevieve Warren <24882762+gewarren@users.noreply.github.com> Date: Fri, 25 Jul 2025 11:39:56 -0700 Subject: [PATCH 07/10] Check for build warnings (#47550) --- .github/copilot-instructions.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 8e8b966ebdb7e..9db9b7e52f184 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -1,3 +1,5 @@ +When you're assigned an issue, after you've completed your work and the workflows (status checks) have run, check to make sure there are no build warnings under the OpenPublishing.Build status check. If there are, open the build report (under View Details) and resolve any build warnings you introduced. + When writing documentation, follow the following guidelines: Unless otherwise specified, all .NET content refers to modern .NET (not .NET Framework). From 3ddc56665694f0fc42eed5724e55055a5ebfe586 Mon Sep 17 00:00:00 2001 From: Genevieve Warren <24882762+gewarren@users.noreply.github.com> Date: Fri, 25 Jul 2025 12:27:54 -0700 Subject: [PATCH 08/10] Fix grammar (#47578) --- .../runtime-libraries/system-reflection-emit-assemblybuilder.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/fundamentals/runtime-libraries/system-reflection-emit-assemblybuilder.md b/docs/fundamentals/runtime-libraries/system-reflection-emit-assemblybuilder.md index 797ec87f91b4d..365915ef93bf1 100644 --- a/docs/fundamentals/runtime-libraries/system-reflection-emit-assemblybuilder.md +++ b/docs/fundamentals/runtime-libraries/system-reflection-emit-assemblybuilder.md @@ -51,7 +51,7 @@ public void CreateAndRunAssembly(string assemblyPath) ## Persisted dynamic assemblies in .NET -The new type derived from the and allows save the dynamic assemblies in .NET Core, check the usage scenarios and examples from [PersistedAssemblyBuilder](system-reflection-emit-persistedassemblybuilder.md) page. +In .NET, the type, which derives from , lets you save dynamic assemblies. For more information, see the usage scenarios and examples at [PersistedAssemblyBuilder](system-reflection-emit-persistedassemblybuilder.md). ## Persisted dynamic assemblies in .NET Framework From 82b22a34dc1b3acfbe63119b121009d0e21841f6 Mon Sep 17 00:00:00 2001 From: Genevieve Warren <24882762+gewarren@users.noreply.github.com> Date: Fri, 25 Jul 2025 16:19:46 -0700 Subject: [PATCH 09/10] Fix command (#47583) --- docs/core/tools/dotnet-tool-install.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/core/tools/dotnet-tool-install.md b/docs/core/tools/dotnet-tool-install.md index eb1f74ed1cfa7..7aba847853310 100644 --- a/docs/core/tools/dotnet-tool-install.md +++ b/docs/core/tools/dotnet-tool-install.md @@ -195,7 +195,7 @@ For more information, see [Install a local tool](global-tools.md#install-a-local Installs [dotnetsay](https://www.nuget.org/packages/dotnetsay/) as a local tool for the current directory. -- **`dotnet tool install -g --verbosity minimal`** +- **`dotnet tool install -g dotnetsay --verbosity minimal`** Installs [dotnetsay](https://www.nuget.org/packages/dotnetsay/) as a global tool with the verbosity of minimal. The default verbosity for global tool is quiet. From cc2d50a24047af44fe8a4e90b76d3b01a00ffb4d Mon Sep 17 00:00:00 2001 From: Genevieve Warren <24882762+gewarren@users.noreply.github.com> Date: Fri, 25 Jul 2025 16:20:09 -0700 Subject: [PATCH 10/10] Remove confusing info (#47506) --- docs/core/tools/dotnet-msbuild.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/core/tools/dotnet-msbuild.md b/docs/core/tools/dotnet-msbuild.md index 1b6a1eea37e69..6d6d51c926c81 100644 --- a/docs/core/tools/dotnet-msbuild.md +++ b/docs/core/tools/dotnet-msbuild.md @@ -25,7 +25,7 @@ The `dotnet msbuild` command allows access to a fully functional MSBuild. The command has the exact same capabilities as the existing MSBuild command-line client for SDK-style projects only. The options are all the same. For more information about the available options, see the [MSBuild command-line reference](/visualstudio/msbuild/msbuild-command-line-reference). -The [dotnet build](dotnet-build.md) command is equivalent to `dotnet msbuild -restore`. When you don't want to build the project and you have a specific target you want to run, use `dotnet build` or `dotnet msbuild` and specify the target. +The [dotnet build](dotnet-build.md) command is equivalent to `dotnet msbuild -restore`. ## Examples