Skip to content

Commit 20b45e5

Browse files
authored
More updates to testing documentation (#45528)
1 parent 5dc0c81 commit 20b45e5

14 files changed

+82
-129
lines changed

docs/core/testing/microsoft-testing-platform-architecture-capabilities.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ ms.date: 07/11/2024
88

99
# Microsoft.Testing.Platform capabilities
1010

11-
In the context of the testing platform, a *capability* refers to the *potential to perform a specific action or provide specific information*. It's a means for the testing framework and extensions to *declare* their *ability* to *operate* in a certain manner or provide specific information to the *requesters*.
11+
In the context of Microsoft.Testing.Platform, a *capability* refers to the *potential to perform a specific action or provide specific information*. It's a means for the testing framework and extensions to *declare* their *ability* to *operate* in a certain manner or provide specific information to the *requesters*.
1212

1313
The *requesters* can be any component involved in a test session, such as the platform, an extension, or the testing framework itself.
1414

@@ -19,7 +19,7 @@ The primary objective of the capability system is to facilitate effective commun
1919
Let's consider a hypothetical example to demonstrate the necessity of a capability system.
2020

2121
> [!NOTE]
22-
> This example is purely for illustrative purposes and isn't currently implemented within the testing platform or any testing framework.
22+
> This example is purely for illustrative purposes and isn't currently implemented within Microsoft.Testing.Platform or any testing framework.
2323
2424
Imagine a situation where you have an extension that requires the testing framework to execute no more than one test at a time. Furthermore, after each test, the extension needs to know the CPU usage for that specific test.
2525

@@ -28,7 +28,7 @@ To accommodate the preceding scenario, you need to inquire from the testing fram
2828
1. It has the capability to execute only one test at a time.
2929
2. It can provide information regarding the amount of CPU consumed by each test.
3030

31-
How can the extension determine if the testing framework has the ability to operate in this mode and provide CPU usage information for a test session? In the testing platform, this capability is represented by an implementation the `Microsoft.Testing.Platform.Capabilities.ICapability` interface:
31+
How can the extension determine if the testing framework has the ability to operate in this mode and provide CPU usage information for a test session? In Microsoft.Testing.Platform, this capability is represented by an implementation the `Microsoft.Testing.Platform.Capabilities.ICapability` interface:
3232

3333
```csharp
3434
// Base capabilities contracts
@@ -111,7 +111,7 @@ In conclusion, let's summarize the primary aspects of the capability system:
111111
* It's essential for facilitating clear and stable communication between components.
112112
* All capabilities should inherit from `ICapability` or an interface that inherits from it, and are exposed through a collection with the `ICapabilities` interface.
113113
* It aids in the evolution of features without causing breaking changes. If a certain capability isn't supported, appropriate action can be taken.
114-
* The responsibility of designing, shipping, and documenting the usage of a capability lies with the *capability owner*. The testing platform can also *own* a capability in the same way as any other extension.
114+
* The responsibility of designing, shipping, and documenting the usage of a capability lies with the *capability owner*. Microsoft.Testing.Platform can also *own* a capability in the same way as any other extension.
115115

116116
## Framework capabilities
117117

docs/core/testing/microsoft-testing-platform-architecture-extensions.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,9 @@ ms.date: 07/11/2024
88

99
# Microsoft.Testing.Platform extensibility
1010

11-
The testing platform consists of a [testing framework](#test-framework-extension) and any number of [extensions](#other-extensibility-points) that can operate *in-process* or *out-of-process*.
11+
Microsoft.Testing.Platform consists of a [testing framework](#test-framework-extension) and any number of [extensions](#other-extensibility-points) that can operate *in-process* or *out-of-process*.
1212

13-
As outlined in the [architecture](./microsoft-testing-platform-architecture.md) section, the testing platform is designed to accommodate a variety of scenarios and extensibility points. The primary and essential extension is undoubtedly the [testing framework](#test-framework-extension) that your tests will utilize. Failing to register this results in startup error. **The [testing framework](#test-framework-extension) is the sole mandatory extension required to execute a testing session.**
13+
As outlined in the [architecture](./microsoft-testing-platform-architecture.md) section, Microsoft.Testing.Platform is designed to accommodate a variety of scenarios and extensibility points. The primary and essential extension is undoubtedly the [testing framework](#test-framework-extension) that your tests will utilize. Failing to register this results in startup error. **The [testing framework](#test-framework-extension) is the sole mandatory extension required to execute a testing session.**
1414

1515
To support scenarios such as generating test reports, code coverage, retrying failed tests, and other potential features, you need to provide a mechanism that allows other extensions to work in conjunction with the [testing framework](#test-framework-extension) to deliver these features not inherently provided by the [testing framework](#test-framework-extension) itself.
1616

@@ -80,7 +80,7 @@ The test framework is the primary extension that provides the testing platform w
8080

8181
### Register a testing framework
8282

83-
This section explains how to register the test framework with the testing platform. You register only one testing framework per test application builder using the `TestApplication.RegisterTestFramework` API as shown in [the testing platform architecture](./microsoft-testing-platform-architecture.md) documentation.
83+
This section explains how to register the test framework with the testing platform. You register only one testing framework per test application builder using the `TestApplication.RegisterTestFramework` API as shown in [Microsoft.Testing.Platform architecture](./microsoft-testing-platform-architecture.md) documentation.
8484

8585
The registration API is defined as follows:
8686

docs/core/testing/microsoft-testing-platform-diagnostics.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,13 @@ ms.author: amauryleve
66
ms.date: 05/30/2024
77
---
88

9-
# Microsoft Testing Platform diagnostics overview
9+
# Microsoft.Testing.Platform diagnostics overview
1010

11-
Microsoft Testing platform analysis ("TPXXX") rules inspect your code for security, performance, design and other issues.
11+
Microsoft.Testing.Platform analysis ("TPXXX") rules inspect your code for security, performance, design and other issues.
1212

1313
## TPEXP
1414

15-
Several APIs of Microsoft Testing Platform are decorated with the <xref:System.Diagnostics.CodeAnalysis.ExperimentalAttribute>. This attribute indicates that the API is experimental and may be removed or changed in future versions of Microsoft Testing Platform. The attribute is used to identify APIs that aren't yet stable and may not be suitable for production use.
15+
Several APIs of Microsoft.Testing.Platform are decorated with the <xref:System.Diagnostics.CodeAnalysis.ExperimentalAttribute>. This attribute indicates that the API is experimental and may be removed or changed in future versions of Microsoft.Testing.Platform. The attribute is used to identify APIs that aren't yet stable and may not be suitable for production use.
1616

1717
To suppress this diagnostic with the `SuppressMessageAttribute`, add the following code to your project:
1818

docs/core/testing/microsoft-testing-platform-extensions-code-coverage.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ ms.date: 04/10/2024
88

99
# Code coverage extensions
1010

11-
This article list and explains all `Microsoft Testing Platform` extensions related to the code coverage capability.
11+
This article lists and explains all Microsoft.Testing.Platform extensions related to the code coverage capability.
1212

1313
You can use the code coverage feature to determine what proportion of your project's code is being tested by coded tests such as unit tests. To effectively guard against bugs, your tests should exercise or *cover* a large proportion of your code.
1414

docs/core/testing/microsoft-testing-platform-extensions-diagnostics.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ ms.date: 04/10/2024
88

99
# Diagnostics extensions
1010

11-
This article list and explains all `Microsoft Testing Platform` extensions related to the diagnostics capability.
11+
This article lists and explains all Microsoft.Testing.Platform extensions related to the diagnostics capability.
1212

1313
## Built-in options
1414

docs/core/testing/microsoft-testing-platform-extensions-hosting.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ ms.date: 04/10/2024
88

99
# Hosting extensions
1010

11-
This article list and explains all `Microsoft Testing Platform` extensions related to the hosting capability.
11+
This article lists and explains all Microsoft.Testing.Platform extensions related to the hosting capability.
1212

1313
## Hot reload
1414

@@ -20,7 +20,7 @@ Hot reload lets you modify your app's managed source code while the application
2020
This extension is shipped as part of the [Microsoft.Testing.Extensions.HotReload](https://nuget.org/packages/Microsoft.Testing.Extensions.HotReload) package.
2121

2222
> [!NOTE]
23-
> The package is shipped with the restrictive Microsoft Testing Platform Tools license.
23+
> The package is shipped with the restrictive Microsoft.Testing.Platform Tools license.
2424
> The full license is available at <https://www.nuget.org/packages/Microsoft.Testing.Extensions.HotReload/1.0.0/License>.
2525
2626
You can easily enable hot reload support by setting the `TESTINGPLATFORM_HOTRELOAD_ENABLED` environment variable to `"1"`.

docs/core/testing/microsoft-testing-platform-extensions-output.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ ms.date: 08/26/2024
88

99
# Output extensions
1010

11-
This article list and explains all **Microsoft Testing Platform** extensions related to the terminal output.
11+
This article lists and explains all Microsoft.Testing.Platform extensions related to the terminal output.
1212

1313
## Terminal test reporter
1414

docs/core/testing/microsoft-testing-platform-extensions-policy.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ ms.date: 04/10/2024
88

99
# Policy extensions
1010

11-
This article list and explains all `Microsoft Testing Platform` extensions related to the policy capability.
11+
This article lists and explains all Microsoft.Testing.Platform extensions related to the policy capability.
1212

1313
## Retry
1414

@@ -19,7 +19,7 @@ This extension is intended for integration tests where the test depends heavily
1919
This extension is shipped as part of [Microsoft.Testing.Extensions.Retry](https://nuget.org/packages/Microsoft.Testing.Extensions.Retry) package.
2020

2121
> [!NOTE]
22-
> The package is shipped with the restrictive Microsoft Testing Platform Tools license.
22+
> The package is shipped with the restrictive Microsoft.Testing.Platform Tools license.
2323
> The full license is available at <https://www.nuget.org/packages/Microsoft.Testing.Extensions.Retry/1.0.0/License>.
2424
2525
The available options are as follows:

docs/core/testing/microsoft-testing-platform-extensions-test-reports.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ ms.date: 04/10/2024
88

99
# Test reports extensions
1010

11-
This article list and explains all `Microsoft Testing Platform` extensions related to the test report capability.
11+
This article lists and explains all Microsoft.Testing.Platform extensions related to the test report capability.
1212

1313
A test report is a file that contains information about the execution and outcome of the tests.
1414

docs/core/testing/microsoft-testing-platform-integration-dotnet-test.md

Lines changed: 12 additions & 103 deletions
Original file line numberDiff line numberDiff line change
@@ -1,119 +1,28 @@
11
---
2-
title: Use Microsoft.Testing.Platform with `dotnet test`
2+
title: Use Microsoft.Testing.Platform with VSTest mode of `dotnet test`
33
description: Learn how to run Microsoft.Testing.Platform tests through dotnet test.
44
author: nohwnd
55
ms.author: jajares
6-
ms.date: 12/20/2023
6+
ms.date: 03/26/2025
77
---
88

9-
# Use Microsoft.Testing.Platform with `dotnet test`
9+
# Use Microsoft.Testing.Platform with VSTest mode of `dotnet test`
1010

11-
This article describes how to use `dotnet test` to run tests when using `Microsoft.Testing.Platform`, and the various options that are available to configure the MSBuild output produced when running tests through Microsoft.Testing.Platform.
11+
This article describes `dotnet test` integration for Microsoft.Testing.Platform that is provided by Microsoft.Testing.Platform.MSBuild when running with VSTest mode of `dotnet test`.
1212

13-
This article shows how to use `dotnet test` to run all tests in a solution (_*.sln_) that uses `Microsoft.Testing.Platform`.
13+
Before reading this article, it's advised to read [Testing with dotnet test](unit-testing-with-dotnet-test.md) first.
1414

15-
## `dotnet test` integration
15+
By default, `dotnet test` is using VSTest behavior to run tests. To enable support for `Microsoft.Testing.Platform` in `dotnet test`, there are two options:
1616

17-
The [dotnet test](../tools/dotnet-test.md) command is a way to run tests from solutions, projects, or already built assemblies. [Microsoft.Testing.Platform](microsoft-testing-platform-intro.md) hooks up into this infrastructure to provide a unified way to run tests, especially when migrating from VSTest to `Microsoft.Testing.Platform`.
17+
1. Use `dotnet test` in VSTest mode, and specify `<TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport>` MSBuild property in your project file.
18+
2. Use `dotnet test` in MTP mode, and enjoy more native support of MTP in `dotnet test`. This is only supported starting with .NET 10 SDK.
1819

19-
### `dotnet test` integration - VSTest mode
20-
21-
`Microsoft.Testing.Platform` provides a [compatibility layer (VSTest Bridge)](./microsoft-testing-platform-extensions-vstest-bridge.md) to work with `dotnet test` seamlessly.
22-
23-
Tests can be run by running:
24-
25-
```dotnetcli
26-
dotnet test
27-
```
28-
29-
This layer runs test through VSTest and integrates with it on VSTest Test Framework Adapter level.
30-
31-
### `dotnet test` - Microsoft.Testing.Platform mode
32-
33-
By default, `dotnet test` is using VSTest behavior to run tests. You can enable support for `Microsoft.Testing.Platform` in `dotnet test` by specifying the `<TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport>` setting in your project file.
34-
35-
```xml
36-
<Project Sdk="Microsoft.NET.Sdk">
37-
38-
<PropertyGroup>
39-
<TargetFramework>net8.0</TargetFramework>
40-
<ImplicitUsings>enable</ImplicitUsings>
41-
<Nullable>enable</Nullable>
42-
43-
<IsPackable>false</IsPackable>
44-
45-
<OutputType>Exe</OutputType>
46-
<EnableMSTestRunner>true</EnableMSTestRunner>
47-
48-
<!-- Add this to your project file. -->
49-
<TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport>
50-
51-
</PropertyGroup>
52-
53-
<!-- ... -->
54-
55-
</Project>
56-
```
57-
58-
> [!NOTE]
59-
> It's highly recommended that you set the `TestingPlatformDotnetTestSupport` property in `Directory.Build.props`. That way, you don't have to add it to every test project file, and you don't risk introducing a new project that doesn't set this property and end up with a solution where some projects are VSTest while others are Microsoft.Testing.Platform, which may not work correctly and is unsupported scenario.
20+
Both of these options are explained in details in the [Testing with dotnet test](unit-testing-with-dotnet-test.md) article.
6021

6122
> [!IMPORTANT]
62-
> Despite `TestingPlatformDotnetTestSupport` being set to `true`, most of the command line options defined in [dotnet test](../tools/dotnet-test.md) remain VSTest oriented and don't impact `Microsoft.Testing.Platform` based tests. To supply arguments to `Microsoft.Testing.Platform`, you need to use one of the methods described in [Microsoft.Testing.Platform command line arguments with dotnet test](#microsofttestingplatform-command-line-arguments-with-dotnet-test).
63-
64-
The list below described all `dotnet test` command line options that are supported by `Microsoft.Testing.Platform`:
65-
66-
- `-a|--arch <ARCHITECTURE>`
67-
- `--artifacts-path <ARTIFACTS_DIR>`
68-
- `-c|--configuration <CONFIGURATION>`
69-
- `-f|--framework <FRAMEWORK>`
70-
- `-e|--environment <NAME="VALUE">`
71-
- `--interactive`
72-
- `--no-build`
73-
- `--nologo`
74-
- `--no-restore`
75-
- `-o|--output <OUTPUT_DIRECTORY>`
76-
- `--os <OS>`
77-
- `-r|--runtime <RUNTIME_IDENTIFIER>`
78-
- `-v|--verbosity <LEVEL>`
79-
80-
These arguments are supported because they are linked to the build step and are independent of the testing platform used.
81-
82-
### `Microsoft.Testing.Platform` command line arguments with `dotnet test`
83-
84-
You can supply arguments that are used to call the testing application in one of the following ways:
85-
86-
- Beginning with `Microsoft.Testing.Platform` version 1.4 (included with MSTest version 3.6), you can add options after the double dash `--` on the command line:
87-
88-
```dotnetcli
89-
dotnet test -- --minimum-expected-tests 10
90-
```
91-
92-
- By using the `TestingPlatformCommandLineArguments` MSBuild property on the command line:
93-
94-
```dotnetcli
95-
dotnet test -p:TestingPlatformCommandLineArguments="--minimum-expected-tests 10"
96-
```
97-
98-
Or in the project file:
99-
100-
```xml
101-
<PropertyGroup>
102-
...
103-
<TestingPlatformCommandLineArguments>--minimum-expected-tests 10</TestingPlatformCommandLineArguments>
104-
</PropertyGroup>
105-
```
106-
107-
## Additional MSBuild options
108-
109-
The MSBuild integration provides options that can be specified in the project file or through global properties on the command line, such as `-p:TestingPlatformShowTestsFailure=true`.
110-
111-
These are the available options:
112-
113-
- [Show failure per test](#show-failure-per-test)
114-
- [Show complete platform output](#show-complete-platform-output)
23+
> The remaining of this article is specific to the first option.
11524
116-
### Show failure per test
25+
## Show failure per test
11726

11827
By default, test failures are summarized into a _.log_ file, and a single failure per test project is reported to MSBuild.
11928

@@ -152,7 +61,7 @@ Or in project file:
15261
</Project>
15362
```
15463

155-
### Show complete platform output
64+
## Show complete platform output
15665

15766
By default, all console output that the underlying test executable writes is captured and hidden from the user. This includes the banner, version information, and formatted test information.
15867

0 commit comments

Comments
 (0)