You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/ai/conceptual/evaluation-libraries.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -18,7 +18,7 @@ The evaluation libraries, which are built on top of the [Microsoft.Extensions.AI
18
18
19
19
## Test integration
20
20
21
-
The libraries are designed to integrate smoothly with existing .NET apps, allowing you to leverage existing testing infrastructures and familiar syntax to evaluate intelligent apps. You can use any test framework (for example, [MSTest](../../core/testing/index.md#mstest), [xUnit](../../core/testing/index.md#xunit), or [NUnit](../../core/testing/index.md#nunit)) and testing workflow (for example, [Test Explorer](/visualstudio/test/run-unit-tests-with-test-explorer), [dotnet test](../../core/tools/dotnet-test.md), or a CI/CD pipeline). The library also provides easy ways to do online evaluations of your application by publishing evaluation scores to telemetry and monitoring dashboards.
21
+
The libraries are designed to integrate smoothly with existing .NET apps, allowing you to leverage existing testing infrastructures and familiar syntax to evaluate intelligent apps. You can use any test framework (for example, [MSTest](../../core/testing/index.md#mstest), [xUnit](../../core/testing/index.md#xunitnet), or [NUnit](../../core/testing/index.md#nunit)) and testing workflow (for example, [Test Explorer](/visualstudio/test/run-unit-tests-with-test-explorer), [dotnet test](../../core/tools/dotnet-test.md), or a CI/CD pipeline). The library also provides easy ways to do online evaluations of your application by publishing evaluation scores to telemetry and monitoring dashboards.
Copy file name to clipboardExpand all lines: docs/core/testing/index.md
+34-17Lines changed: 34 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -35,47 +35,64 @@ Keep in mind there are [best practices](unit-testing-best-practices.md) for writ
35
35
36
36
## Testing tools
37
37
38
-
.NET is a multi-language development platform, and you can write various test types for [C#](../../csharp/index.yml), [F#](../../fsharp/index.yml), and [Visual Basic](../../visual-basic/index.yml). For each of these languages, you can choose between several test frameworks.
38
+
When running tests in .NET, there are two components involved: the test platform and the test framework.
39
39
40
-
### xUnit
40
+
### Test platforms
41
41
42
-
[xUnit](https://xunit.net) is a free, open-source, community-focused unit testing tool for .NET. The original inventor of NUnit v2 wrote xUnit.net. xUnit.net is the latest technology for unit testing .NET apps. It also works with ReSharper, CodeRush, and TestDriven.NET. xUnit.net is a project of the [.NET Foundation](https://dotnetfoundation.org) and operates under its code of conduct.
42
+
The test platform is the engine that runs the tests and acts as a communication channel with IDEs. For example, Visual Studio can send a discovery request to the test platform so that it can display the available tests in Test Explorer. The test platform responds back to the IDE with the tests it found. Similar communication happens for test execution.
43
+
44
+
VSTest has been used for many years in .NET and was the only test platform in the ecosystem. Early in 2024, the first stable version of a new test platform, called [Microsoft.Testing.Platform (MTP)](./unit-testing-platform-intro.md), was released.
45
+
46
+
### Test frameworks
47
+
48
+
The test framework is built on top of the test platform. It defines the set of attributes and APIs that are available to you, as a test author. It's usually powered by a test adapter, which acts as a communication layer between the test framework and the test platform. The popular test frameworks are MSTest, NUnit, TUnit, and xUnit.net.
49
+
50
+
#### MSTest
51
+
52
+
[MSTest](https://github.com/microsoft/testfx) is the Microsoft test framework for all .NET languages. It's extensible and works with .NET CLI, Visual Studio, Visual Studio Code, and Rider. It supports both VSTest and Microsoft.Testing.Platform.
43
53
44
54
For more information, see the following resources:
45
55
46
-
-[Unit testing with C#](unit-testing-with-dotnet-test.md)
47
-
-[Unit testing with F#](unit-testing-fsharp-with-dotnet-test.md)
48
-
-[Unit testing with Visual Basic](unit-testing-visual-basic-with-dotnet-test.md)
56
+
-[Microsoft.Testing.Platform support in MSTest (MSTest runner)](unit-testing-mstest-runner-intro.md)
57
+
-[Unit testing with C#](unit-testing-with-mstest.md)
58
+
-[Unit testing with F#](unit-testing-fsharp-with-mstest.md)
59
+
-[Unit testing with Visual Basic](unit-testing-visual-basic-with-mstest.md)
49
60
50
-
### NUnit
61
+
####NUnit
51
62
52
-
[NUnit](https://nunit.org) is a unit-testing framework for all .NET languages. Initially, NUnit was ported from JUnit, and the current production release has been rewritten with many new features and support for a wide range of .NET platforms. It's a project of the [.NET Foundation](https://dotnetfoundation.org).
63
+
[NUnit](https://nunit.org) is a unit-testing framework for all .NET languages. Initially, NUnit was ported from JUnit, and the current production release has been rewritten with many new features and support for a wide range of .NET platforms. It's a project of the [.NET Foundation](https://dotnetfoundation.org). It supports both VSTest and Microsoft.Testing.Platform.
53
64
54
65
For more information, see the following resources:
55
66
67
+
-[Microsoft.Testing.Platform support in NUnit (NUnit runner)](unit-testing-nunit-runner-intro.md)
56
68
-[Unit testing with C#](unit-testing-with-nunit.md)
57
69
-[Unit testing with F#](unit-testing-fsharp-with-nunit.md)
58
70
-[Unit testing with Visual Basic](unit-testing-visual-basic-with-nunit.md)
59
71
60
-
###MSTest
72
+
#### TUnit
61
73
62
-
[MSTest](https://github.com/microsoft/testfx) is the Microsoft test framework for all .NET languages. It's extensible and works with both .NET CLI and Visual Studio. For more information, see the following resources:
74
+
[TUnit](https://thomhurst.github.io/TUnit/) is entirely built on top of Microsoft.Testing.Platform and doesn't support VSTest. For more information, refer to TUnit documentation.
63
75
64
-
-[Unit testing with C#](unit-testing-with-mstest.md)
65
-
-[Unit testing with F#](unit-testing-fsharp-with-mstest.md)
66
-
-[Unit testing with Visual Basic](unit-testing-visual-basic-with-mstest.md)
76
+
#### xUnit.net
77
+
78
+
[xUnit.net](https://xunit.net) is a free, open-source, community-focused unit testing tool for .NET. The original inventor of NUnit v2 wrote xUnit.net. xUnit.net is the latest technology for unittesting .NET apps. It also works with ReSharper, CodeRush, and TestDriven.NET. xUnit.net is a project of the [.NET Foundation](https://dotnetfoundation.org) and operates under its code of conduct. It supports both VSTest and Microsoft.Testing.Platform
67
79
68
-
#### MSTest runner
80
+
For more information, see the following resources:
81
+
82
+
-[Microsoft.Testing.Platform support in xUnit.net v3](https://xunit.net/docs/getting-started/v3/microsoft-testing-platform)
83
+
-[Unit testing with C#](unit-testing-with-dotnet-test.md)
84
+
-[Unit testing with F#](unit-testing-fsharp-with-dotnet-test.md)
85
+
-[Unit testing with Visual Basic](unit-testing-visual-basic-with-dotnet-test.md)
69
86
70
-
The MSTest runner is a lightweight and portable alternative to [VSTest](https://github.com/microsoft/vstest) for running tests in continuous integration (CI) pipelines, and in Visual Studio Test Explorer. For more information, see [MSTest runner overview](unit-testing-mstest-runner-intro.md).
87
+
## Running tests
71
88
72
89
### .NET CLI
73
90
74
-
You can run a solutions unit test from the [.NET CLI](../tools/index.md) with the [dotnet test](../tools/dotnet-test.md) command. The .NET CLI exposes most of the functionality that [Integrated Development Environments (IDEs)](#ide) make available through user interfaces. The .NET CLI is cross-platform and available to use as part of continuous integration and delivery pipelines. The .NET CLI is used with scripted processes to automate common tasks.
91
+
You can run unit tests from all test projects in a solution using the [.NET CLI](../tools/index.md) with the [dotnet test](../tools/dotnet-test.md) command. The .NET CLI exposes most of the functionality that [Integrated Development Environments (IDEs)](#ide) make available through user interfaces. The .NET CLI is cross-platform and available to use as part of continuous integration and delivery pipelines. The .NET CLI is used with scripted processes to automate common tasks.
75
92
76
93
### IDE
77
94
78
-
Whether you're using Visual Studio or Visual Studio Code, there are graphical user interfaces for testing functionality. There are more features available to IDEs than the CLI, for example, [Live Unit Testing](/visualstudio/test/live-unit-testing). For more information, see [Including and excluding tests with Visual Studio](/visualstudio/test/live-unit-testing#include-and-exclude-test-projects-and-test-methods).
95
+
Whether you're using Visual Studio, Visual Studio Code, or Rider, there are graphical user interfaces for testing functionality. There are more features available to IDEs than the CLI, for example, [Live Unit Testing](/visualstudio/test/live-unit-testing). For more information, see [Including and excluding tests with Visual Studio](/visualstudio/test/live-unit-testing#include-and-exclude-test-projects-and-test-methods).
Copy file name to clipboardExpand all lines: docs/core/testing/unit-testing-mstest-runner-intro.md
+19-11Lines changed: 19 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,16 +1,16 @@
1
1
---
2
-
title: MSTest runner overview
2
+
title: Microsoft.Testing.Platform support in MSTest (MSTest runner)
3
3
description: Learn about the MSTest runner, a lightweight way to run tests without depending on the .NET SDK.
4
4
author: nohwnd
5
5
ms.author: jajares
6
6
ms.date: 12/15/2023
7
7
---
8
8
9
-
# MSTest runner overview
9
+
# Microsoft.Testing.Platform support in MSTest (MSTest runner)
10
10
11
-
The MSTest runner is a lightweight and portable alternative to [VSTest](https://github.com/microsoft/vstest)for running tests in all contexts (for example, continuous integration (CI) pipelines, CLI, Visual Studio Test Explorer, and VS Code Test Explorer). The MSTest runner is embedded directly in your MSTest test projects, and there are no other app dependencies, such as `vstest.console` or `dotnet test`, needed to run your tests.
11
+
MSTest supports running tests with both VSTest and [Microsoft.Testing.Platform (MTP)](./unit-testing-platform-intro.md). The support for MTP is powered by the MSTest runner, which can run tests in all contexts (for example, continuous integration (CI) pipelines, CLI, Visual Studio Test Explorer, and VS Code Text Explorer). The MSTest runner is embedded directly in your MSTest test projects, and there are no other app dependencies, such as `vstest.console` or `dotnet test`, needed to run your tests. However, you can still run your tests using `dotnet test`.
12
12
13
-
The MSTest runner is open source, and builds on a[`Microsoft.Testing.Platform`](./unit-testing-platform-intro.md) library. You can find `Microsoft.Testing.Platform` code in [microsoft/testfx](https://github.com/microsoft/testfx/tree/main/src/Platform/Microsoft.Testing.Platform) GitHub repository. The MSTest runner comes bundled with `MSTest in 3.2.0-preview.23623.1` or newer.
13
+
The MSTest runner is open source and builds on the[`Microsoft.Testing.Platform`](./unit-testing-platform-intro.md) library. You can find `Microsoft.Testing.Platform` code in the [microsoft/testfx](https://github.com/microsoft/testfx/tree/main/src/Platform/Microsoft.Testing.Platform) GitHub repository. The MSTest runner comes bundled with `MSTest in 3.2.0` or newer.
14
14
15
15
## Enable MSTest runner in an MSTest project
16
16
@@ -19,7 +19,7 @@ It's recommended to use [MSTest SDK](./unit-testing-mstest-sdk.md) as it greatly
19
19
When you use `MSTest SDK`, by default you're opted in to using MSTest runner.
20
20
21
21
```xml
22
-
<ProjectSdk="MSTest.Sdk/3.3.1">
22
+
<ProjectSdk="MSTest.Sdk/3.8.2">
23
23
24
24
<PropertyGroup>
25
25
<TargetFramework>net8.0</TargetFramework>
@@ -30,7 +30,7 @@ When you use `MSTest SDK`, by default you're opted in to using MSTest runner.
30
30
</Project>
31
31
```
32
32
33
-
Alternatively, you can enable MSTest runner by adding the `EnableMSTestRunner` property and setting `OutputType` to `Exe` in your project file. You also need to ensure that you're using `MSTest 3.2.0-preview.23623.1` or newer.
33
+
Alternatively, you can enable MSTest runner by adding the `EnableMSTestRunner` property and setting `OutputType` to `Exe` in your project file. You also need to ensure that you're using `MSTest 3.2.0` or newer. We strongly recommend you update to the latest MSTest version available.
34
34
35
35
Consider the following example project file:
36
36
@@ -40,14 +40,19 @@ Consider the following example project file:
40
40
<PropertyGroup>
41
41
<!-- Enable the MSTest runner, this is an opt-in feature -->
Coverlet collector isn't compatible with MSTest runner, you can
@@ -75,7 +83,7 @@ Consider the following example project file:
75
83
```
76
84
77
85
> [!TIP]
78
-
> It's advised to set the `EnableMSTestRunner`property in *Directory.Build.props* file instead of *csproj* file to ensure all test projects in your solution are using the MSTest runner.
86
+
> To ensure all test projects in your solution use the MSTest runner, set the `EnableMSTestRunner`and `TestingPlatformDotnetTestSupport` properties in *Directory.Build.props* file instead of individual project files.
Copy file name to clipboardExpand all lines: docs/core/testing/unit-testing-nunit-runner-intro.md
+19-11Lines changed: 19 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,20 +1,23 @@
1
1
---
2
-
title: NUnit runner overview
2
+
title: Microsoft.Testing.Platform support in NUnit (NUnit runner)
3
3
description: Learn about the NUnit runner, a lightweight way to run tests without depending on the .NET SDK.
4
4
author: Evangelink
5
5
ms.author: amauryleve
6
6
ms.date: 05/21/2024
7
7
---
8
8
9
-
# NUnit runner overview
9
+
# Microsoft.Testing.Platform support in NUnit (NUnit runner)
10
10
11
-
The NUnit runner is a lightweight and portable alternative to [VSTest](https://github.com/microsoft/vstest)for running tests in all contexts (for example, continuous integration (CI) pipelines, CLI, Visual Studio Test Explorer, and VS Code Text Explorer). The NUnit runner is embedded directly in your NUnit test projects, and there are no other app dependencies, such as `vstest.console` or `dotnet test`, needed to run your tests.
11
+
NUnit supports running tests with both VSTest and [Microsoft.Testing.Platform (MTP)](./unit-testing-platform-intro.md). The support for MTP is powered by the NUnit runner, which can run tests in all contexts (for example, continuous integration (CI) pipelines, CLI, Visual Studio Test Explorer, and VS Code Text Explorer). The NUnit runner is embedded directly in your NUnit test projects, and there are no other app dependencies, such as `vstest.console` or `dotnet test`, needed to run your tests. However, you can still run your tests using `dotnet test`.
12
12
13
-
The NUnit runner is open source, and builds on a [`Microsoft.Testing.Platform`](./unit-testing-platform-intro.md) library. You can find `Microsoft.Testing.Platform` code in [microsoft/testfx](https://github.com/microsoft/testfx/tree/main/src/Platform/Microsoft.Testing.Platform) GitHub repository. The NUnit runner comes bundled with `NUnit 5.0.0-beta.2` or newer.
13
+
The NUnit runner is open source, and builds on top of [`Microsoft.Testing.Platform`](./unit-testing-platform-intro.md). You can find `Microsoft.Testing.Platform` code in [microsoft/testfx](https://github.com/microsoft/testfx/tree/main/src/Platform/Microsoft.Testing.Platform) GitHub repository. The NUnit runner is supported in NUnit3TestAdapter version 5.0 or greater. For more information, see [NUnit and Microsoft.Testing.Platform](https://docs.nunit.org/articles/vs-test-adapter/NUnit-And-Microsoft-Test-Platform.html)
14
14
15
15
## Enable NUnit runner in a NUnit project
16
16
17
-
You can enable NUnit runner by adding the `EnableNUnitRunner` property and setting `OutputType` to `Exe` in your project file. You also need to ensure that you're using `NUnit 5.0.0-beta.2` or newer.
17
+
You can enable NUnit runner by adding the `EnableNUnitRunner` property and setting `OutputType` to `Exe` in your project file. You also need to ensure that you're using `NUnit3TestAdapter` version 5.0 or newer.
18
+
19
+
> [!TIP]
20
+
> To ensure all test projects in your solution use the NUnit runner, set the `EnableNUnitRunner` and `TestingPlatformDotnetTestSupport` properties in *Directory.Build.props* file instead of individual project files.
18
21
19
22
Consider the following example project file:
20
23
@@ -24,24 +27,29 @@ Consider the following example project file:
24
27
<PropertyGroup>
25
28
<!-- Enable the NUnit runner, this is an opt-in feature -->
![dotnet-policy-service[bot]](https://avatars.githubusercontent.com/in/285228?v=4&size=40){kind=avatar}
0 commit comments