Skip to content

Add better error messages for dotnet test with file/directory arguments in Microsoft Testing Platform (MTP) #50108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented Aug 7, 2025

When users run dotnet test with file or directory arguments as positional parameters, they previously received confusing MSBuild or VSTest errors. This PR adds validation to detect these patterns and provide helpful guidance for Microsoft Testing Platform (MTP) users.

Problem

Users often try commands like:

  • dotnet test MySolution.sln
  • dotnet test MyProject.csproj
  • dotnet test path/to/directory
  • dotnet test test.dll

These would result in confusing errors like:

  • "Solution file error MSB5010: No file format header found"
  • "Project file could not be loaded. Data at the root level is invalid"
  • "Specify a project or solution file. The current working directory does not contain a project or solution file"

Solution

Added validation in the Microsoft Testing Platform command path (TestingPlatformCommand.cs) to detect file/directory arguments and suggest using the appropriate flags directly (--solution, --project, --directory, --test-modules).

Examples

Before:

$ dotnet test MySolution.sln
MySolution.sln : Solution file error MSB5010: No file format header found.

After (when MTP is enabled):

$ dotnet test MySolution.sln
Solution file 'MySolution.sln' was provided as a positional argument.
Use the --solution flag: dotnet test --solution MySolution.sln

Implementation Details

  • Validation only applies when Microsoft Testing Platform is enabled
  • Validation triggers for files that actually exist to avoid interfering with MSBuild's normal error handling
  • Non-breaking change - existing functionality preserved for VSTest path
  • Comprehensive test coverage for all validation scenarios in the MTP command path
  • VSTest path (TestCommand.cs) remains unchanged to preserve existing behavior

Fixes #49710.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

@Copilot Copilot AI changed the title [WIP] Better error message for missing --solution, --project, --directory, or --test-modules in dotnet test Add better error messages for dotnet test with file/directory arguments Aug 7, 2025
@Copilot Copilot AI requested a review from Youssef1313 August 7, 2025 10:20
Copilot finished work on behalf of Youssef1313 August 7, 2025 10:20
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is for VSTest. We need to update the implementation for MTP instead, which is in TestingPlatformCommand.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot This file is for VSTest. We need to update the implementation for MTP instead, which is in TestingPlatformCommand.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're absolutely correct. I've now implemented the validation logic in TestingPlatformCommand.cs as well to ensure consistent error messages regardless of which test runner is being used. The implementation provides appropriate guidance for MTP users to use the correct flags (--solution, --project, --directory, --test-modules) when they provide file/directory arguments as positional parameters. Commit 455bd3b addresses this by adding the validation to both command paths.

… for file/directory arguments

Co-authored-by: Youssef1313 <[email protected]>
@Copilot Copilot AI changed the title Add better error messages for dotnet test with file/directory arguments Add better error messages for dotnet test with file/directory arguments in both VSTest and MTP Aug 7, 2025
@Copilot Copilot AI requested a review from Youssef1313 August 7, 2025 16:13
Copilot finished work on behalf of Youssef1313 August 7, 2025 16:13
Copy link
Member

@Youssef1313 Youssef1313 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot Revert all changes in src/Cli/dotnet/Commands/Test/TestCommand.cs

Copy link
Contributor Author

Copilot AI commented Aug 8, 2025

@copilot Revert all changes in src/Cli/dotnet/Commands/Test/TestCommand.cs

All changes in TestCommand.cs have been reverted. Commit 8c23e19.

@Copilot Copilot AI changed the title Add better error messages for dotnet test with file/directory arguments in both VSTest and MTP Add better error messages for dotnet test with file/directory arguments in Microsoft Testing Platform (MTP) Aug 8, 2025
@Copilot Copilot AI requested a review from Youssef1313 August 8, 2025 15:36
Copilot finished work on behalf of Youssef1313 August 8, 2025 15:36
@Youssef1313 Youssef1313 marked this pull request as ready for review August 8, 2025 17:22
@Copilot Copilot AI review requested due to automatic review settings August 8, 2025 17:22
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds validation for Microsoft Testing Platform (MTP) to provide better error messages when users run dotnet test with file or directory arguments as positional parameters instead of using the appropriate command-line flags.

  • Validates file/directory arguments and suggests using proper flags (--solution, --project, --directory, --test-modules)
  • Adds comprehensive test coverage for all validation scenarios in the MTP command path
  • Updates localization files with new error messages

Reviewed Changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/Cli/dotnet/Commands/Test/ValidationUtility.cs Adds new validation method to detect file/directory arguments and provide helpful error messages
src/Cli/dotnet/Commands/Test/TestingPlatformCommand.cs Integrates validation into the MTP command execution path
src/Cli/dotnet/Commands/CliCommandStrings.resx Adds English resource strings for the new error messages
src/Cli/dotnet/Commands/xlf/*.xlf Updates localization files with new error message entries marked as "new"
test/dotnet.Tests/CommandTests/Test/TestCommandValidationTests.cs Adds comprehensive test coverage for validation scenarios

<value>Specifying a directory for 'dotnet test' should be via '--directory'.</value>
</data>
<data name="TestCommandUseTestModules" xml:space="preserve">
<value>Specifying a dlls or executables for 'dotnet test' should be via '--test-modules'.</value>
Copy link
Preview

Copilot AI Aug 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grammar error: 'a dlls' should be 'dlls' (remove the article 'a' before the plural noun).

Suggested change
<value>Specifying a dlls or executables for 'dotnet test' should be via '--test-modules'.</value>
<value>Specifying dlls or executables for 'dotnet test' should be via '--test-modules'.</value>

Copilot uses AI. Check for mistakes.

<value>Specifying a directory for 'dotnet test' should be via '--directory'.</value>
</data>
<data name="TestCommandUseTestModules" xml:space="preserve">
<value>Specifying a dlls or executables for 'dotnet test' should be via '--test-modules'.</value>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot Please update "Specifying a dlls" to "Specifying dlls" (remove the "a") in the resx and all xlf files.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the grammar error by removing "a" from "Specifying a dlls" in the .resx file and all .xlf files. Commit 5d6eab3.

Copilot finished work on behalf of Youssef1313 August 9, 2025 03:31
@Copilot Copilot AI requested a review from Youssef1313 August 9, 2025 03:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Better error message for missing --solution, --project, --directory, or --test-modules in dotnet test
2 participants