From fd6abb4cd43768e6bda9f0fb8d09d5be9229da65 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 10 Oct 2025 17:10:03 +0000 Subject: [PATCH 1/4] Initial plan From 76eecbb85d61c4ece126da65875147e0aaeb9d2b Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 10 Oct 2025 17:15:56 +0000 Subject: [PATCH 2/4] Add code coverage support to Arcade SDK with VSTest integration Co-authored-by: rmarinho <1235097+rmarinho@users.noreply.github.com> --- Documentation/ArcadeSdk.md | 108 ++++++++++++++++++ .../tools/Tests.props | 35 ++++++ .../tools/Tests.targets | 1 + .../tools/VSTest.targets | 73 ++++++++++++ 4 files changed, 217 insertions(+) diff --git a/Documentation/ArcadeSdk.md b/Documentation/ArcadeSdk.md index a5df38e66b2..0e931ff053d 100644 --- a/Documentation/ArcadeSdk.md +++ b/Documentation/ArcadeSdk.md @@ -1064,6 +1064,114 @@ To override the default Shared Framework version that is selected based on the t Timeout to apply to an individual invocation of the test runner (e.g. `xunit.console.exe`) for a single configuration. Integer number of milliseconds. +### Code Coverage Properties + +Arcade SDK supports code coverage collection when using the VSTest runner. To enable code coverage, set `UseVSTestRunner` to `true` and `CollectCoverage` to `true`. + +#### `CollectCoverage` (bool) + +Set to `true` to enable code coverage collection when running tests. Default is `false`. + +```text +msbuild Project.UnitTests.csproj /t:Test /p:UseVSTestRunner=true /p:CollectCoverage=true +``` + +Or set it in your project file: + +```xml + + true + true + +``` + +#### `CodeCoverageFormat` (string) + +Specifies the output format for code coverage reports. Supported values: `cobertura`, `opencover`, `lcov`, `json`, or a combination like `cobertura,opencover`. Default is `cobertura`. + +The Cobertura format is supported by Azure DevOps and can be published using the `PublishCodeCoverageResults` task. + +#### `CodeCoverageOutputDirectory` (string) + +Directory where code coverage reports will be generated. Default is `$(ArtifactsTestResultsDir)coverage`. + +#### `CoverageDeterministic` (bool) + +Enable or disable generation of deterministic coverage reports. Default is `true`. + +#### `CoverageInclude` (string) + +Semicolon-separated list of assembly patterns to include in code coverage. Uses glob patterns. Default is empty (includes all assemblies). + +Example: +```xml +[MyProject]*;[MyLibrary]* +``` + +#### `CoverageExclude` (string) + +Semicolon-separated list of assembly patterns to exclude from code coverage. Uses glob patterns. Default is empty. + +Example: +```xml +[*.Tests]*;[xunit.*]* +``` + +#### `CoverageIncludeByFile` (string) + +Semicolon-separated list of file path patterns to include in code coverage. Uses glob patterns. Default is empty. + +#### `CoverageExcludeByFile` (string) + +Semicolon-separated list of file path patterns to exclude from code coverage. Uses glob patterns. Default is empty. + +Example: +```xml +**/*Designer.cs;**/Generated/*.cs +``` + +#### `CoverageExcludeByAttribute` (string) + +Semicolon-separated list of attributes to exclude from code coverage. Default is empty. + +Example: +```xml +Obsolete;GeneratedCode;CompilerGenerated +``` + +#### Example: Complete Code Coverage Configuration + +```xml + + + net8.0 + true + true + cobertura + [*.Tests]*;[xunit.*]* + **/*Designer.cs + + + + + + +``` + +Then run tests with: +```text +./build.sh --test +``` + +Code coverage reports will be generated in `artifacts/TestResults/coverage/` directory in Cobertura format, which can be published to Azure DevOps using the `PublishCodeCoverageResults` task in your pipeline: + +```yaml +- task: PublishCodeCoverageResults@2 + inputs: + summaryFileLocation: '$(Build.SourcesDirectory)/artifacts/TestResults/coverage/**/coverage.cobertura.xml' + codecoverageTool: 'cobertura' +``` + ### `GenerateResxSource` (bool) When set to `true`, Arcade will generate a class source for all embedded .resx files. diff --git a/src/Microsoft.DotNet.Arcade.Sdk/tools/Tests.props b/src/Microsoft.DotNet.Arcade.Sdk/tools/Tests.props index 25f911cb753..ad035ae3786 100644 --- a/src/Microsoft.DotNet.Arcade.Sdk/tools/Tests.props +++ b/src/Microsoft.DotNet.Arcade.Sdk/tools/Tests.props @@ -36,4 +36,39 @@ + + + + false + + + cobertura + + + $(ArtifactsTestResultsDir)coverage + + + true + + + + + + + + + + + + + + + + + + + + + + diff --git a/src/Microsoft.DotNet.Arcade.Sdk/tools/Tests.targets b/src/Microsoft.DotNet.Arcade.Sdk/tools/Tests.targets index 8ae93d3051f..756ede01728 100644 --- a/src/Microsoft.DotNet.Arcade.Sdk/tools/Tests.targets +++ b/src/Microsoft.DotNet.Arcade.Sdk/tools/Tests.targets @@ -73,6 +73,7 @@ $(TestRunnerAdditionalArguments) $(RunArguments) $(RunCommand) + $(CollectCoverage) diff --git a/src/Microsoft.DotNet.Arcade.Sdk/tools/VSTest.targets b/src/Microsoft.DotNet.Arcade.Sdk/tools/VSTest.targets index 6aba0addf1a..c7c88cb53be 100644 --- a/src/Microsoft.DotNet.Arcade.Sdk/tools/VSTest.targets +++ b/src/Microsoft.DotNet.Arcade.Sdk/tools/VSTest.targets @@ -27,6 +27,7 @@ <_TestRunnerCommand>"$(DotNetTool)" test $(_TestAssembly) --logger:"console%3Bverbosity=normal" --logger:"trx%3BLogFileName=$(_TestResultTrxFileName)" --logger:"html%3BLogFileName=$(_TestResultHtmlFileName)" "--ResultsDirectory:$(_TestResultDirectory)" "--Framework:%(TestToRun.TargetFrameworkIdentifier),Version=%(TestToRun.TargetFrameworkVersion)" <_TestRunnerCommand Condition="'%(TestToRun.TestRunSettingsFile)' != ''">$(_TestRunnerCommand) "--settings:%(TestToRun.TestRunSettingsFile)" + <_TestRunnerCommand Condition="'%(TestToRun.CollectCoverage)' == 'true'">$(_TestRunnerCommand) --collect:"XPlat Code Coverage" <_TestRunnerCommand Condition="'$(_TestRunnerAdditionalArguments)' != ''">$(_TestRunnerCommand) $(_TestRunnerAdditionalArguments) + + + + <_GeneratedRunSettingsFile>$(ArtifactsObjDir)$(MSBuildProjectName).runsettings + + + <_CoverageIncludeFilter Condition="'$(CoverageInclude)' != ''">$([System.String]::new('$(CoverageInclude)').Replace(';',',')) + + + <_CoverageExcludeFilter Condition="'$(CoverageExclude)' != ''">$([System.String]::new('$(CoverageExclude)').Replace(';',',')) + + + <_CoverageIncludeByFileFilter Condition="'$(CoverageIncludeByFile)' != ''">$([System.String]::new('$(CoverageIncludeByFile)').Replace(';',',')) + + + <_CoverageExcludeByFileFilter Condition="'$(CoverageExcludeByFile)' != ''">$([System.String]::new('$(CoverageExcludeByFile)').Replace(';',',')) + + + <_CoverageExcludeByAttributeFilter Condition="'$(CoverageExcludeByAttribute)' != ''">$([System.String]::new('$(CoverageExcludeByAttribute)').Replace(';',',')) + + + <_RunSettingsContent> + + + + + + + $(_CodeCoverageFormat) + $(_CodeCoverageOutputPath) + $(_CoverageDeterministic) + $(_CoverageIncludeFilter) + $(_CoverageExcludeFilter) + + $(_CoverageExcludeByFileFilter) + false + $(_CoverageExcludeByAttributeFilter) + true + DoesNotReturnAttribute + false + true + + + + +]]> + + + + + + + + $(_GeneratedRunSettingsFile) + + + + + + + $(VSTestRunSettingsFile) + + + <_CodeCoverageFormat Condition="'$(CollectCoverage)' == 'true'">$(CodeCoverageFormat) + <_CodeCoverageOutputPath Condition="'$(CollectCoverage)' == 'true'">$(CodeCoverageOutputDirectory) + <_CoverageDeterministic Condition="'$(CollectCoverage)' == 'true'">$(CoverageDeterministic) From 4703bcb3efa18eced5e53abb5abc67ed73ded43b Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 10 Oct 2025 17:22:50 +0000 Subject: [PATCH 3/4] Fix runsettings generation to use ItemGroup for proper XML formatting Co-authored-by: rmarinho <1235097+rmarinho@users.noreply.github.com> --- .../tools/VSTest.targets | 57 +++++++++---------- 1 file changed, 28 insertions(+), 29 deletions(-) diff --git a/src/Microsoft.DotNet.Arcade.Sdk/tools/VSTest.targets b/src/Microsoft.DotNet.Arcade.Sdk/tools/VSTest.targets index c7c88cb53be..9b7f84946f7 100644 --- a/src/Microsoft.DotNet.Arcade.Sdk/tools/VSTest.targets +++ b/src/Microsoft.DotNet.Arcade.Sdk/tools/VSTest.targets @@ -104,39 +104,38 @@ <_CoverageExcludeByAttributeFilter Condition="'$(CoverageExcludeByAttribute)' != ''">$([System.String]::new('$(CoverageExcludeByAttribute)').Replace(';',',')) - - - <_RunSettingsContent> - - - - - - - $(_CodeCoverageFormat) - $(_CodeCoverageOutputPath) - $(_CoverageDeterministic) - $(_CoverageIncludeFilter) - $(_CoverageExcludeFilter) - - $(_CoverageExcludeByFileFilter) - false - $(_CoverageExcludeByAttributeFilter) - true - DoesNotReturnAttribute - false - true - - - - -]]> - + + + <_RunSettingsLines Include="<?xml version="1.0" encoding="utf-8"?>" /> + <_RunSettingsLines Include="<RunSettings>" /> + <_RunSettingsLines Include=" <DataCollectionRunSettings>" /> + <_RunSettingsLines Include=" <DataCollectors>" /> + <_RunSettingsLines Include=" <DataCollector friendlyName="XPlat code coverage">" /> + <_RunSettingsLines Include=" <Configuration>" /> + <_RunSettingsLines Include=" <Format>$(_CodeCoverageFormat)</Format>" /> + <_RunSettingsLines Include=" <OutputPath>$(_CodeCoverageOutputPath)</OutputPath>" /> + <_RunSettingsLines Include=" <DeterministicReport>$(_CoverageDeterministic)</DeterministicReport>" /> + <_RunSettingsLines Include=" <Include>$(_CoverageIncludeFilter)</Include>" Condition="'$(_CoverageIncludeFilter)' != ''" /> + <_RunSettingsLines Include=" <Exclude>$(_CoverageExcludeFilter)</Exclude>" Condition="'$(_CoverageExcludeFilter)' != ''" /> + <_RunSettingsLines Include=" <ExcludeByFile>$(_CoverageExcludeByFileFilter)</ExcludeByFile>" Condition="'$(_CoverageExcludeByFileFilter)' != ''" /> + <_RunSettingsLines Include=" <IncludeTestAssembly>false</IncludeTestAssembly>" /> + <_RunSettingsLines Include=" <ExcludeByAttribute>$(_CoverageExcludeByAttributeFilter)</ExcludeByAttribute>" Condition="'$(_CoverageExcludeByAttributeFilter)' != ''" /> + <_RunSettingsLines Include=" <SkipAutoProps>true</SkipAutoProps>" /> + <_RunSettingsLines Include=" <DoesNotReturnAttribute>DoesNotReturnAttribute</DoesNotReturnAttribute>" /> + <_RunSettingsLines Include=" <SingleHit>false</SingleHit>" /> + <_RunSettingsLines Include=" <UseSourceLink>true</UseSourceLink>" /> + <_RunSettingsLines Include=" </Configuration>" /> + <_RunSettingsLines Include=" </DataCollector>" /> + <_RunSettingsLines Include=" </DataCollectors>" /> + <_RunSettingsLines Include=" </DataCollectionRunSettings>" /> + <_RunSettingsLines Include="</RunSettings>" /> + + From 02a26bdc3c2e24558142354b3ba043202b210471 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 10 Oct 2025 17:24:00 +0000 Subject: [PATCH 4/4] Add code coverage example documentation Co-authored-by: rmarinho <1235097+rmarinho@users.noreply.github.com> --- Documentation/CodeCoverageExample.md | 108 +++++++++++++++++++++++++++ 1 file changed, 108 insertions(+) create mode 100644 Documentation/CodeCoverageExample.md diff --git a/Documentation/CodeCoverageExample.md b/Documentation/CodeCoverageExample.md new file mode 100644 index 00000000000..6252a4181a5 --- /dev/null +++ b/Documentation/CodeCoverageExample.md @@ -0,0 +1,108 @@ +# Code Coverage Example for Arcade SDK + +This example demonstrates how to enable code coverage collection in test projects using Arcade SDK. + +## Prerequisites + +- Arcade SDK 11.0.0 or later +- Test project using XUnit + +## Basic Configuration + +Add the following properties to your test project file: + +```xml + + + net8.0 + + + true + + + true + + + cobertura + + +``` + +## Running Tests with Coverage + +Run tests using the build script: + +```bash +./build.sh --test +``` + +Or using MSBuild directly: + +```bash +dotnet build /t:Test /p:Configuration=Release +``` + +## Coverage Reports + +Code coverage reports will be generated in the `artifacts/TestResults/coverage/` directory. + +For Cobertura format, the report file will be named `coverage.cobertura.xml` and can be published to Azure DevOps. + +## Azure DevOps Integration + +Add the following task to your Azure Pipelines YAML to publish coverage results: + +```yaml +- task: PublishCodeCoverageResults@2 + inputs: + summaryFileLocation: '$(Build.SourcesDirectory)/artifacts/TestResults/coverage/**/coverage.cobertura.xml' + codecoverageTool: 'cobertura' + displayName: 'Publish Code Coverage Results' +``` + +## Advanced Configuration + +### Filtering Coverage + +Exclude specific assemblies or files from coverage: + +```xml + + + [*.Tests]*;[xunit.*]*;[Moq]* + + + **/*Designer.cs;**/Generated/*.cs + + + Obsolete;GeneratedCode;CompilerGenerated + +``` + +### Multiple Output Formats + +Generate coverage in multiple formats: + +```xml + + cobertura,opencover,lcov + +``` + +## Troubleshooting + +### Coverage not collected + +- Ensure `UseVSTestRunner` is set to `true` +- Verify `CollectCoverage` is set to `true` +- Check that `coverlet.collector` package is restored (should be automatic) + +### Empty coverage report + +- Make sure tests are actually running and passing +- Verify the test project has a reference to the code you want to cover + +## See Also + +- [Arcade SDK Documentation](../Documentation/ArcadeSdk.md) +- [Coverlet Documentation](https://github.com/coverlet-coverage/coverlet)