Not getting crash reports on iOS or Android

[Darcy3000s]

Looking for some guidance on where I might have gone wrong

Trying to integrate the BugSplat plugin with Unreal Engine 5.5.1

I've included the BugSplat plugin in our engine plugins folder, and ran into a few issues:

#1 Source/BugSplat/private/BugSplat.cpp needs to include PropertyEditorModule - it's not currently including it, and was causing build failures

#2 Android Symbols weren't uploading:
the UPL is adding a task to run after:
task.name == 'ue4CompleteDebug' ||
task.name == 'ue4CompleteRelease' ||
task.name == 'ueCompleteDebug' ||
task.name == 'ueCompleteRelease'

But when we run our CI build, there is no Gradle task with that name - I have included
task.name == 'ueCompleteBundleDebug' ||
task.name == 'ueCompleteBundleRelease'
As that's the closest task naming I could find to the task BugSplat is trying to run after

After that, the Symbol upload would fail because the directory the plugin is searching for Symbols in was empty - it was searching in:
$S(BuildDir)/gradle/app/build/intermediates/merged_native_libs/release
But our CI pipeline isn't outputting anything to that location - the relevant files are actually at this location for us:
$S(BuildDir)/../gradle/app/build/intermediates/merged_native_libs/release

BugSplat is checking whether we're running "Shipping" config to decide whether to upload symbols from the release or debug configuration - but this seems wrong, I believe UE is using Release for Development & Test builds (These are the build configurations we're using)

#3 iOS Symbols weren't uploading:
I've uploaded the mac uploader script to use this targetName.dSYM directory instead of the one at latest (targetName.app.dSYM), because the .app.dsym file doesn't exist in our output binaries
export dsymFileSpec="$targetName.app.dSYM"
export dsymFileSpec="$targetName.dSYM"

#4 iOS is crashing on launch - After all this iOS crashes on launch, unfortunately with no crash report (I don't currently have access to a Mac to get a crash dump from XCode)

#5 Android builds don't upload crash reports - now that Android is building, and symbols are uploading, I can see the Android versions in the BugSplat dashboard, but when we crash (either using the "debug crash" command, or through legitimate game crashes) nothing is reported to the BugSplat dashboard

I'm not sure what I've done wrong here - any guidance?

[bobbyg603]

Hi @Darcy3000s, sorry you've running into these issues! Are you using a source or a launcher build? The plugin should go in the game's Plugin folder (you can see an example in our my-unreal-crasher sample).

1.) I'm not sure how we haven't run into this. we'll get it fixed. thanks for reporting the issue

2.) Our plugin hasn't been tested in a CI environment, it's been developed to work on developer machines interactively. How does your CI system work? Are you building to a custom location? I'm also very puzzled about the gradle task mismatch as I tested this recently on my system with 5.5.3 and everything seemed to work. I will review the gradle tasks again and see if we can make a fix there.

3.) I haven't forgotten about this, we're going to relax the pattern match so that we match *.dSYM instead of .app.dSYM. Still don't understand why your build doesn't have the .app.dSYM extension but seems easy enough to work around.

4.) I've struggled with this as well. It usually boils down to Unreal not copying the framework into the build directory and/or the framework not making it onto the device. I'm happy to look into this further, but would need at least a log statement at the time of the crash in order to proceed.

5.) What device are you using? Many older devices are not supported. Another thing to check, if you connect to the device via the adb shell and find the deployment path pm path com.company.game, does it contain libcrashpad_handler.so, libstdc++.so, and libcurl.so? Also debug crash causes android to raise SIGTRAP which Crashpad does not catch. This is a known issue that we're working on. Something like volatile char *ptr = (char*)0xC0FFEEC0; *ptr += 1; as seen in our sample should get caught.

[Darcy3000s]

Hey @bobbyg603 we're building our games against UE source
If we put the plugin in the games plugin folder, then we won't be able to get symbolicated crash logs for UE Editor crashes - since we won't have uploaded editor symbols
The UE Crash reporter does include a symbolicated log of the current callstack of the crashing thread, but that sometimes may not be enough to diagnose a crash, it would be better to include the editor symbols so we can get the callstack of all threads

On point 1, it's worth noting that Unreal Engine uses "Unity Builds" by default, which may be why you haven't run into it, there may have been an indirect dependency in your blob
Additionally, we build using IWYU (https://dev.epicgames.com/documentation/en-us/unreal-engine/include-what-you-use-iwyu-for-unreal-engine-programming) not sure what your test set up uses

2.) Ah that's a bit unfortunate to hear, any plans on testing this against CI environments? We currently use UBT for our CI builds, but will likely switch over in the near future to use BuildGraph instead

3.) Great - I also don't understand really why our build outputs would differ, but I'd guess building from editor vs UBT command line might have something to do with it

4.) Ah interesting - I could try throwing the plugin in the project rather than the engine, in case there's some incorrect pathing caused by that difference, but it's not ideal (See the point about editor crash reports)

5.) I'll follow up with the team member who's been testing Android crashes for me and get them to test a proper crash rather than debug crash, thanks!

Any chance if I DM my BugSplat details to you that we could look at resetting the trial period - I believe it lapses today, and we haven't been able to get this working yet

[bobbyg603]

Hey @Darcy3000s thanks for following up here. Happy to reset your trial, please send an email to support@bugsplat.com.

I believe the current version of the plugin will actually upload the Editor symbols for source builds (at least it was doing so on my system with the plugin folder in the Game directory when I authored this issue). We were going to turn this off because we thought it was undesirable but it sounds like we should handle this via an option! However, I'm not sure this will meaningfully change the behavior of any of the points you listed, but it is one thing I noticed that was different.

1. Thanks for the tip, we'll read up and hopefully learn something!

2. We would love to support CI environments! If you can continue to help us to help you this is an area we're eager to improve. Perhaps you could include details about how you invoke UBT in an email to support@bugsplat.com?

3. Send us your full UBT and we'll be sure to test it!

4. Although we've had success building with XCode on our development systems we're also chasing an issue that we were just made aware of about nested frameworks not being supported on iOS. This seems to have caused at least one other person to crash at start up. Perhaps this works when building from XCode but not when building with UBT? There is active development here so stay tuned for updates.

5. Please also send us device info when you get a chance! We've had success with Pixel 8a and Quest 3. If your team member wants to reach out to us directly there's a few easy things we can do to narrow the scope of the issue.

Thank you for bringing all of these issues to our attention! Mobile crash reporting for Unreal is tough, but we'll get this all fixed soon 🙂

[bobbyg603]

Whoops, accidentally closed this meant to close #113