Device/integration tests for mobile on .net10

[jamescrosswell]

Resolves #4744

#skip-changelog

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.85%. Comparing base (ac97aef) to head (ed8b558).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4750      +/-   ##
==========================================
- Coverage   73.85%   73.85%   -0.01%     
==========================================
  Files         485      485              
  Lines       17689    17689              
  Branches     3497     3497              
==========================================
- Hits        13065    13064       -1     
  Misses       3762     3762              
- Partials      862      863       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jamescrosswell]

Tricky... locally everything passes for me:

However in CI we've got 2 tests failing in Debug and 2 tests failing in Release:

...
2025-11-20T03:50:51.4905577Z ##[error][-] MAUI app (net10.0, Release).Native crash (Release) 21.13s (21.12s|3ms)�[0m
2025-11-20T03:50:51.4916597Z ##[group]Message�[0m
2025-11-20T03:50:52.4056556Z Expected a collection with size 1, but got collection with size 25
2025-11-20T03:50:52.7693005Z at <ScriptBlock>, /home/runner/work/sentry-dotnet/sentry-dotnet/integration-test/android.Tests.ps1:170�[0m
...
025-11-20T03:53:44.9710461Z ##[error][-] MAUI app (net10.0, Debug).Java crash (Debug) 10.9s (10.9s|1ms)�[0m
2025-11-20T03:53:44.9723309Z ##[group]Message�[0m,
2025-11-20T03:53:45.0186641Z Expected a collection with size 1, but got collection with size 2
2025-11-20T04:01:08.0804390Z at $result.Envelopes() | Should -HaveCount 1, /home/runner/work/sentry-dotnet/sentry-dotnet/integration-test/android.Tests.ps1:156�[0m
...
2025-11-20T03:58:20.3428226Z ##[error][-] MAUI app (net10.0, Release).Native crash (Release) 22.23s (22.23s|3ms)�[0m
2025-11-20T03:58:20.3429986Z ##[group]Message�[0m
2025-11-20T03:58:21.0057104Z Expected a collection with size 1, but got collection with size 28
2025-11-20T03:50:52.7693005Z at <ScriptBlock>, /home/runner/work/sentry-dotnet/sentry-dotnet/integration-test/android.Tests.ps1:170�[0m
...
2025-11-20T04:01:08.0369087Z ##[error][-] MAUI app (net10.0, Debug).Java crash (Debug) 9.43s (9.43s|1ms)�[0m
2025-11-20T04:01:08.0387771Z ##[group]Message�[0m
2025-11-20T04:01:08.0683949Z Expected a collection with size 1, but got collection with size 2
2025-11-20T04:01:08.0804390Z at $result.Envelopes() | Should -HaveCount 1, /home/runner/work/sentry-dotnet/sentry-dotnet/integration-test/android.Tests.ps1:156�[0m
...

Looking at the 2 envelopes received for the Java crashes (java_received.json) the only difference is the timestamp:

... so it looks like we're getting duplicate envelopes being sent for some reason
(but only on CI of course) 😕

[jamescrosswell]

@supervacuus wondering if you might have some insights. When bumping to .NET 10, our Android integration tests fail when creating a native crash...

2025-11-26T00:27:34.1540247Z Expected a collection with size 1, but got collection with size 16

Basically we have a single crash, so we're expecting a single envelope... but instead we get loads of envelopes (from sentry.native). We don't see the same behaviour in .net 9 (those tests pass fine)... so maybe a change in behaviour in terms of how .net 10 wraps signals 🤷🏻 although, if so, not one that's documented in the release notes. It's not always 16 envelopes - in previous runs it's been 26 or 28 etc. so basically just "way more than expected".

I can't reproduce this locally - it only happens to us in CI (which makes it pretty tricky to analyse).

It looks like maybe there are some tests that are already relevant in the native repo, so I created a PR to bump these to .NET 10 to see if we see the same issue there... although those are run on Linux Alpine (not Android) so not necessarily expecting the same results:

• Bump dotnet signal tests to .net 10 sentry-native#1457

[supervacuus]

Hi @jamescrosswell 👋

Basically we have a single crash, so we're expecting a single envelope... but instead we get loads of envelopes (from sentry.native). We don't see the same behaviour in .net 9 (those tests pass fine)... so maybe a change in behaviour in terms of how .net 10 wraps signals 🤷🏻 although, if so, not one that's documented in the release notes. It's not always 16 envelopes - in previous runs it's been 26 or 28 etc. so basically just "way more than expected".

From what I have seen over the last year in the .NET runtime's "documentation" on signal usage, I would be surprised if a change in behavior in that area made its way into the release notes.

I don't have an immediate idea and it is rather hard to follow for me, because i don't know what your test does and which of those outputs are relevant.

However, intuitively, a variable number of envelopes rather than one looks like a signal loop (typically caused by an abort(), because it resets the signal mask before it raises a SIGABRT), where some function after serializing the envelope causes a crash in the handler.

But it's really hard to say for me from what I know about your tests. It would be interesting to see all envelopes rather than just their count.

I can't reproduce this locally - it only happens to us in CI (which makes it pretty tricky to analyse).

But does it repro on CI stably? Would it be possible to actions/upload-artifact the entire envelope cache? The JSON printed in the logs only shows a stackrace with a SIGILL and a single frame from the crash() function libsentrysupplemental.so.

It looks like maybe there are some tests that are already relevant in the native repo, so I created a PR to bump these to .NET 10 to see if we see the same issue there... although those are run on Linux Alpine (not Android) so not necessarily expecting the same results:

getsentry/sentry-native#1457

Yes, the .NET tests fail with that bump, but likely due to .NET 10 being an unsupported target (see my response).

I would also ping @jpnurmi on this topic.

[jamescrosswell]

I would also ping @jpnurmi on this topic.

Yes normally. He's unfortunately unavailable until next year 😢

I don't have an immediate idea and it is rather hard to follow for me, because i don't know what your test does and which of those outputs are relevant.

That's fair... JP actually wrote the tests. They're supposed to be high level simple integration tests in pester/powershell that compile a .net app and run it, passing in an argument that determines what the app should do. For example, in the test that's failing here, the argument is "Native", which just means the app should throw a native exception. That will crash the app, so the test then runs the app (which crashes) then runs the app again (to ensure any crashes dumped to disk get handled) then checks to make sure an envelope containing the expected event was sent/received by the fake sentry server.

a variable number of envelopes rather than one looks like a signal loop (typically caused by an abort(), because it resets the signal mask before it raises a SIGABRT), where some function after serializing the envelope causes a crash in the handler.

Hm, that gives me some clues. After each test, the app gets shut down with this. That should shut down Sentry before calling KillProcess, but I think I see an issue with the implementation for cascading shutdown to other SDKs here... in the case of Android and iOS there is no cascading so the Java SDK and it's embedded Native SDK don't get explicitly shutdown.

I'll dig into that and report back - thanks @supervacuus 🙏🏻

[jamescrosswell]

So I tried cascading the SDK shutdown to the Java SDK:

sentry-dotnet/src/Sentry/Internal/Hub.cs

Line 874 in 278c04c

JavaSdk.Sentry.Close();

Unfortunately the Java SDK also doesn't cascade this to the instance of the native SDK 😢
https://github.com/getsentry/sentry-java/blob/fc5ccaf5844ae029d93c566b6fd1b4f5c2e4da83/sentry/src/main/java/io/sentry/Sentry.java#L715-L723

I guess it would be possible to rectify that in the Java SDK but that would require a series of new releases etc. I'll see if I can find an alternative workaround.

[jamescrosswell]

@Flash0ver in the latest test runs we're seeing:

Error: [-] MAUI app (net10.0, Release).Null reference exception (Release) 7.58s (7.57s|4ms)
Expected string '"type":"SIGSEGV"' to match no elements in collection

However when I run the same code in a sample application, I don't see the same behaviour, so I'm pretty sure this is as a result of the difficulty in finding a way to shut the app down in a way that does't create a signal loop and lots of other unrelated exceptions that get captured by the native client.

One possible "proper fix" would be to cascades the SDK shutdown logic, but I don't want to delay the version 6 release in order to build that so I think we park this PR for the time being and circle back post v6 release (perhaps in the new year when JP is back on deck).