Enable more tests to run on all 3 runtimes, part 6 #10595
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
grendello
changed the title
grendello
force-pushed
the
dev/grendel/test-runtimes-6
branch
from
71c0fc0 to
4d78971
Compare
Contributor
Author
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
grendello
force-pushed
the
dev/grendel/test-runtimes-6
branch
2 times, most recently
from
81a53cd to
79bbe97
Compare
Contributor
Author
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Contributor
Author
|
/azp run |
Contributor
Author
|
Please CI deities, make it not time out this time? |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Member
|
I think |
Contributor
Author
The odd thing is that this test doesn't fail locally: |
Contributor
Author
|
And this is the reason why, it's <PropertyGroup>
<!--
Use $(TreatWarningsAsErrors) for CI builds. We have grandfathered some projects that have existing warnings,
but we would like to go ahead and prevent any other projects from getting new warnings.
This can be opted into locally with $(_AndroidTreatWarningsAsErrors) = true.
-->
<_AllowProjectWarnings Condition=" '$(MSBuildProjectFile)' == 'jnienv-gen.csproj' ">true</_AllowProjectWarnings>
<_AllowProjectWarnings Condition=" '$(MSBuildProjectFile)' == 'Mono.Android.csproj' ">true</_AllowProjectWarnings>
<_AllowProjectWarnings Condition=" '$(MSBuildProjectFile)' == 'Mono.Android.NET-Tests.csproj' ">true</_AllowProjectWarnings>
<_AllowProjectWarnings Condition=" '$(MSBuildProjectFile)' == 'MSBuildDeviceIntegration.csproj' ">true</_AllowProjectWarnings>
<_AllowProjectWarnings Condition=" '$(MSBuildProjectFile)' == 'NativeAOT.csproj' ">true</_AllowProjectWarnings>
<_AllowProjectWarnings Condition=" '$(MSBuildProjectFile)' == 'TestRunner.Core.NET.csproj' ">true</_AllowProjectWarnings>
<_AllowProjectWarnings Condition=" '$(MSBuildProjectFile)' == 'Xamarin.Android.Build.Tests.csproj' ">true</_AllowProjectWarnings>
<_AllowProjectWarnings Condition=" '$(MSBuildProjectFile)' == 'Xamarin.Android.JcwGen-Tests.csproj' ">true</_AllowProjectWarnings>
<_AllowProjectWarnings Condition=" '$(MSBuildProjectFile)' == 'Xamarin.Android.NUnitLite.NET.csproj' ">true</_AllowProjectWarnings>
<_AllowProjectWarnings Condition=" '$(MSBuildProjectFile)' == 'Xamarin.ProjectTools.csproj' ">true</_AllowProjectWarnings>
<TreatWarningsAsErrors Condition=" ('$(RunningOnCI)' == 'true' OR '$(_AndroidTreatWarningsAsErrors)' == 'true') AND '$(_AllowProjectWarnings)' != 'true' ">true</TreatWarningsAsErrors>
</PropertyGroup> |
grendello
force-pushed
the
dev/grendel/test-runtimes-6
branch
3 times, most recently
from
0f1fb8a to
8ec7d61
Compare
jonathanpeppers
approved these changes
Dec 4, 2025
grendello
force-pushed
the
dev/grendel/test-runtimes-6
branch
3 times, most recently
from
024cc4e to
74d6990
Compare
grendello
force-pushed
the
dev/grendel/test-runtimes-6
branch
from
74d6990 to
4e8003c
Compare
grendello
force-pushed
the
dev/grendel/test-runtimes-6
branch
2 times, most recently
from
5768795 to
058b806
Compare
grendello
force-pushed
the
dev/grendel/test-runtimes-6
branch
from
058b806 to
2562ea9
Compare
jonathanpeppers
approved these changes
Jan 16, 2026
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
NOTE: should be merged after #10585
Summary
This pull request enhances the test coverage and flexibility of the Xamarin.Android build tests by parameterizing tests to run across different Android runtimes and refactoring the way build properties are handled for NativeAOT scenarios. The changes ensure that tests are more robust and adaptable to future runtime changes, and they improve warning management—especially for CI environments.
Test Parameterization and Runtime Handling
AssetPackTests.csandCodeBehindTests.csto use the[Values]attribute for bothisReleaseandAndroidRuntime runtimeparameters, enabling comprehensive testing across all runtime configurations.IgnoreUnsupportedConfiguration, preventing test failures on invalid configurations.Build Properties and Warning Management
GetBuildPropertiesto dynamically construct theNoWarnproperty, including additional IL and XA warnings when running NativeAOT tests on CI, improving test reliability and output clarity.Test Infrastructure Improvements
TestProjectInfoclass to track the selected runtime and test root directory, supporting better test isolation and reporting.runtimeparameter through all test helpers and runners.These changes collectively make the test suite more maintainable and future-proof, especially as new runtimes or build scenarios are introduced.