-
Notifications
You must be signed in to change notification settings - Fork 6.1k
Document MSTEST0047 and the MSTest 4.3 analyzers (MSTEST0064-0068) #54187
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Evangelink
wants to merge
3
commits into
main
Choose a base branch
from
dev/amauryleve/mstest-analyzers-43
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| --- | ||
| title: "MSTEST0047: Unused TestContext parameter suppressor" | ||
| description: "Learn about code suppressor MSTEST0047: Unused TestContext parameter suppressor." | ||
| ms.date: 06/04/2026 | ||
| f1_keywords: | ||
| - MSTEST0047 | ||
| - UnusedParameterSuppressor | ||
| helpviewer_keywords: | ||
| - IDE0060 | ||
| - UnusedParameterSuppressor | ||
| - MSTEST0047 | ||
| author: Evangelink | ||
| ms.author: amauryleve | ||
| ai-usage: ai-assisted | ||
| --- | ||
| # MSTEST0047: Unused `TestContext` parameter suppressor | ||
|
|
||
| | Property | Value | | ||
| |-------------------------------------|--------------------------------------------------------------------------------| | ||
| | **Rule ID** | MSTEST0047 | | ||
| | **Title** | Suppress IDE0060 for `TestContext` parameter on initialize and cleanup methods | | ||
| | **Category** | Suppressor | | ||
| | **Introduced in version** | 4.2.0 | | ||
|
|
||
| ## Suppressor description | ||
|
|
||
| Suppress the [IDE0060: Remove unused parameter](../../../fundamentals/code-analysis/style-rules/ide0060.md) diagnostic for the `TestContext` parameter on the following MSTest fixture methods: | ||
|
|
||
| - A method marked with <xref:Microsoft.VisualStudio.TestTools.UnitTesting.AssemblyInitializeAttribute>. | ||
| - A method marked with <xref:Microsoft.VisualStudio.TestTools.UnitTesting.ClassInitializeAttribute>. | ||
| - A method marked with `GlobalTestInitializeAttribute` (available starting with MSTest 4.3). | ||
| - A method marked with `GlobalTestCleanupAttribute` (available starting with MSTest 4.3). | ||
|
|
||
| These signatures are required by the test framework even when the `TestContext` parameter isn't used in the method body, so IDE0060 isn't actionable on them. | ||
|
|
||
| ## When to disable suppressor | ||
|
|
||
| .NET suppressors cannot be disabled. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| --- | ||
| title: "MSTEST0064: Prefer async assertion methods" | ||
| description: "Learn about code analysis rule MSTEST0064: Prefer async assertion methods" | ||
| ms.date: 06/04/2026 | ||
| f1_keywords: | ||
| - MSTEST0064 | ||
| - PreferAsyncAssertionAnalyzer | ||
| helpviewer_keywords: | ||
| - PreferAsyncAssertionAnalyzer | ||
| - MSTEST0064 | ||
| author: Evangelink | ||
| ms.author: amauryleve | ||
| ai-usage: ai-assisted | ||
| dev_langs: | ||
| - CSharp | ||
| --- | ||
| # MSTEST0064: Prefer async assertion methods | ||
|
|
||
| | Property | Value | | ||
| |-------------------------------------|----------------------------------------------------| | ||
| | **Rule ID** | MSTEST0064 | | ||
| | **Title** | Prefer async assertion methods | | ||
| | **Category** | Usage | | ||
| | **Fix is breaking or non-breaking** | Non-breaking | | ||
| | **Enabled by default** | Yes | | ||
| | **Default severity** | Info | | ||
| | **Introduced in version** | 4.3.0 | | ||
| | **Is there a code fix** | No | | ||
|
|
||
| > [!NOTE] | ||
| > This rule is available starting with MSTest 4.3. | ||
|
|
||
| ## Cause | ||
|
|
||
| A test method uses <xref:Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Throws*?displayProperty=nameWithType> or <xref:Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ThrowsExactly*?displayProperty=nameWithType> to assert an exception is thrown by code that is asynchronous and blocks the asynchronous operation using `GetAwaiter().GetResult()`. | ||
|
|
||
| ## Rule description | ||
|
|
||
| When asserting an exception is thrown by asynchronous code, prefer the async assertion methods `Assert.ThrowsAsync` and `Assert.ThrowsExactlyAsync` over blocking the asynchronous operation with `GetAwaiter().GetResult()`. Blocking on async code can cause deadlocks in some synchronization contexts and is harder to read than the equivalent `await`-based assertion. | ||
|
|
||
| ```csharp | ||
| [TestClass] | ||
| public class TestClass | ||
| { | ||
| [TestMethod] | ||
| public async Task Test_ThrowsOnAsyncCall() | ||
| { | ||
| // Violation: blocks the async call inside Assert.Throws. | ||
| Assert.Throws<InvalidOperationException>(() => DoAsync().GetAwaiter().GetResult()); | ||
| } | ||
|
|
||
| private static async Task DoAsync() | ||
| { | ||
| await Task.Yield(); | ||
| throw new InvalidOperationException(); | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## How to fix violations | ||
|
|
||
| Use the asynchronous assertion method and `await` it: | ||
|
|
||
| ```csharp | ||
| [TestClass] | ||
| public class TestClass | ||
| { | ||
| [TestMethod] | ||
| public async Task Test_ThrowsOnAsyncCall() | ||
| { | ||
| await Assert.ThrowsAsync<InvalidOperationException>(() => DoAsync()); | ||
| } | ||
|
|
||
| private static async Task DoAsync() | ||
| { | ||
| await Task.Yield(); | ||
| throw new InvalidOperationException(); | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| The same applies to <xref:Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ThrowsExactly*?displayProperty=nameWithType>, which has an `Assert.ThrowsExactlyAsync` counterpart. | ||
|
|
||
| ## When to suppress warnings | ||
|
|
||
| You can suppress this warning when the containing test method cannot be made `async` (for example, a synchronous test overload required by a base class or interface signature) and you need to keep the blocking call. | ||
|
|
||
| ## Suppress a warning | ||
|
|
||
| If you just want to suppress a single violation, add preprocessor directives to your source file to disable and then re-enable the rule. | ||
|
|
||
| ```csharp | ||
| #pragma warning disable MSTEST0064 | ||
| // The code that's violating the rule is on this line. | ||
| #pragma warning restore MSTEST0064 | ||
| ``` | ||
|
|
||
| To disable the rule for a file, folder, or project, set its severity to `none` in the [configuration file](../../../fundamentals/code-analysis/configuration-files.md). | ||
|
|
||
| ```ini | ||
| [*.{cs,vb}] | ||
| dotnet_diagnostic.MSTEST0064.severity = none | ||
| ``` | ||
|
|
||
| For more information, see [How to suppress code analysis warnings](../../../fundamentals/code-analysis/suppress-warnings.md). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,101 @@ | ||||||
| --- | ||||||
| title: "MSTEST0065: Avoid Assert.AreEqual on collection types" | ||||||
| description: "Learn about code analysis rule MSTEST0065: Avoid Assert.AreEqual on collection types" | ||||||
| ms.date: 06/04/2026 | ||||||
| f1_keywords: | ||||||
| - MSTEST0065 | ||||||
| - AvoidAssertAreEqualOnCollectionsAnalyzer | ||||||
| helpviewer_keywords: | ||||||
| - AvoidAssertAreEqualOnCollectionsAnalyzer | ||||||
| - MSTEST0065 | ||||||
| author: Evangelink | ||||||
| ms.author: amauryleve | ||||||
| ai-usage: ai-assisted | ||||||
| dev_langs: | ||||||
| - CSharp | ||||||
| --- | ||||||
| # MSTEST0065: Avoid `Assert.AreEqual` on collection types | ||||||
|
|
||||||
| | Property | Value | | ||||||
| |-------------------------------------|----------------------------------------------------| | ||||||
| | **Rule ID** | MSTEST0065 | | ||||||
| | **Title** | Avoid `Assert.AreEqual` on collection types | | ||||||
| | **Category** | Usage | | ||||||
| | **Fix is breaking or non-breaking** | Non-breaking | | ||||||
| | **Enabled by default** | Yes | | ||||||
| | **Default severity** | Warning | | ||||||
| | **Introduced in version** | 4.3.0 | | ||||||
| | **Is there a code fix** | No | | ||||||
|
|
||||||
| > [!NOTE] | ||||||
| > This rule is available starting with MSTest 4.3. | ||||||
|
|
||||||
| ## Cause | ||||||
|
|
||||||
| A call to <xref:Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual*?displayProperty=nameWithType> or <xref:Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreNotEqual*?displayProperty=nameWithType> is made on a value whose static type implements <xref:System.Collections.Generic.IEnumerable%601> (other than <xref:System.String>). | ||||||
|
|
||||||
| ## Rule description | ||||||
|
|
||||||
| <xref:Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual*> and <xref:Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreNotEqual*> use <xref:System.Collections.Generic.EqualityComparer%601.Default?displayProperty=nameWithType>. For most collection types — for example arrays, <xref:System.Collections.Generic.List%601>, or any user-defined type implementing <xref:System.Collections.Generic.IEnumerable%601> — this falls back to reference equality (or to whatever equality the type defines for itself), and not to element-wise comparison. As a result, the assertion almost never asserts what the test author intended. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| ```csharp | ||||||
| [TestClass] | ||||||
| public class TestClass | ||||||
| { | ||||||
| [TestMethod] | ||||||
| public void Test() | ||||||
| { | ||||||
| var expected = new[] { 1, 2, 3 }; | ||||||
| var actual = new[] { 1, 2, 3 }; | ||||||
|
|
||||||
| // Violation: this compares references, not contents, and fails. | ||||||
| Assert.AreEqual(expected, actual); | ||||||
| } | ||||||
| } | ||||||
| ``` | ||||||
|
|
||||||
| ## How to fix violations | ||||||
|
|
||||||
| Choose the assertion that matches your intent: | ||||||
|
|
||||||
| - Use `Assert.AreSequenceEqual` for ordered element-wise comparison. | ||||||
| - Use `Assert.AreSequenceEqual(expected, actual, SequenceOrder.InAnyOrder)` for unordered element-wise comparison. | ||||||
| - Use `Assert.AreEquivalent` for deep structural comparison. | ||||||
|
|
||||||
| ```csharp | ||||||
| [TestClass] | ||||||
| public class TestClass | ||||||
| { | ||||||
| [TestMethod] | ||||||
| public void Test() | ||||||
| { | ||||||
| var expected = new[] { 1, 2, 3 }; | ||||||
| var actual = new[] { 1, 2, 3 }; | ||||||
|
|
||||||
| Assert.AreSequenceEqual(expected, actual); | ||||||
| } | ||||||
| } | ||||||
| ``` | ||||||
|
|
||||||
| ## When to suppress warnings | ||||||
|
|
||||||
| Suppress this warning only if the type defines its own `Equals`/`GetHashCode` to compare contents and you intentionally rely on that. | ||||||
|
|
||||||
| ## Suppress a warning | ||||||
|
|
||||||
| If you just want to suppress a single violation, add preprocessor directives to your source file to disable and then re-enable the rule. | ||||||
|
|
||||||
| ```csharp | ||||||
| #pragma warning disable MSTEST0065 | ||||||
| // The code that's violating the rule is on this line. | ||||||
| #pragma warning restore MSTEST0065 | ||||||
| ``` | ||||||
|
|
||||||
| To disable the rule for a file, folder, or project, set its severity to `none` in the [configuration file](../../../fundamentals/code-analysis/configuration-files.md). | ||||||
|
|
||||||
| ```ini | ||||||
| [*.{cs,vb}] | ||||||
| dotnet_diagnostic.MSTEST0065.severity = none | ||||||
| ``` | ||||||
|
|
||||||
| For more information, see [How to suppress code analysis warnings](../../../fundamentals/code-analysis/suppress-warnings.md). | ||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| --- | ||
| title: "MSTEST0066: '[Ignore]' should specify a justification" | ||
| description: "Learn about code analysis rule MSTEST0066: '[Ignore]' should specify a justification" | ||
| ms.date: 06/04/2026 | ||
| f1_keywords: | ||
| - MSTEST0066 | ||
| - IgnoreShouldHaveJustificationAnalyzer | ||
| helpviewer_keywords: | ||
| - IgnoreShouldHaveJustificationAnalyzer | ||
| - MSTEST0066 | ||
| author: Evangelink | ||
| ms.author: amauryleve | ||
| ai-usage: ai-assisted | ||
| dev_langs: | ||
| - CSharp | ||
| --- | ||
| # MSTEST0066: `[Ignore]` should specify a justification | ||
|
|
||
| | Property | Value | | ||
| |-------------------------------------|----------------------------------------------------| | ||
| | **Rule ID** | MSTEST0066 | | ||
| | **Title** | `[Ignore]` should specify a justification | | ||
| | **Category** | Design | | ||
| | **Fix is breaking or non-breaking** | Non-breaking | | ||
| | **Enabled by default** | Yes | | ||
| | **Default severity** | Info | | ||
| | **Introduced in version** | 4.3.0 | | ||
| | **Is there a code fix** | No | | ||
|
|
||
| > [!NOTE] | ||
| > This rule is available starting with MSTest 4.3. | ||
|
|
||
| ## Cause | ||
|
|
||
| A test method or test class is marked with <xref:Microsoft.VisualStudio.TestTools.UnitTesting.IgnoreAttribute> but no justification message is provided. | ||
|
|
||
| ## Rule description | ||
|
|
||
| To improve the discoverability and triage of skipped tests, an `[Ignore]` attribute applied to a test method or test class should include a non-empty message explaining why the test or class is ignored. A justification message helps reviewers understand the intent and prevents tests from being silently disabled forever. | ||
|
|
||
| ```csharp | ||
| [TestClass] | ||
| public class TestClass | ||
| { | ||
| [TestMethod] | ||
| [Ignore] // Violation - no justification provided. | ||
| public void Test() | ||
| { | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## How to fix violations | ||
|
|
||
| Pass a non-empty message to the `[Ignore]` attribute, either positionally or via the `IgnoreMessage` named property: | ||
|
|
||
| ```csharp | ||
| [TestClass] | ||
| public class TestClass | ||
| { | ||
| [TestMethod] | ||
| [Ignore("Disabled until issue #1234 is fixed.")] | ||
| public void Test() | ||
| { | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ```csharp | ||
| [TestClass] | ||
| [Ignore(IgnoreMessage = "Disabled until issue #1234 is fixed.")] | ||
| public class TestClass | ||
| { | ||
| [TestMethod] | ||
| public void Test() | ||
| { | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## When to suppress warnings | ||
|
|
||
| Don't suppress warnings from this rule. Ignored tests without a justification are easy to forget. | ||
|
|
||
| ## Suppress a warning | ||
|
|
||
| If you just want to suppress a single violation, add preprocessor directives to your source file to disable and then re-enable the rule. | ||
|
|
||
| ```csharp | ||
| #pragma warning disable MSTEST0066 | ||
| // The code that's violating the rule is on this line. | ||
| #pragma warning restore MSTEST0066 | ||
| ``` | ||
|
|
||
| To disable the rule for a file, folder, or project, set its severity to `none` in the [configuration file](../../../fundamentals/code-analysis/configuration-files.md). | ||
|
|
||
| ```ini | ||
| [*.{cs,vb}] | ||
| dotnet_diagnostic.MSTEST0066.severity = none | ||
| ``` | ||
|
|
||
| For more information, see [How to suppress code analysis warnings](../../../fundamentals/code-analysis/suppress-warnings.md). |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.