feat: minimal tombstone integration (#4933)

* feat: minimal tombstone integration (disabled by default, options internal)

* redo a seemingly missed spotlessApply

* use non-plural option + mark integration also as internal + update API

* init size exceptions since we know we only need one

Co-authored-by: Markus Hintersteiner <[email protected]>

* move `instructionAddressAdjustment` to `SentryStackTrace`

* add copyright notice to tombstone.proto

* add historical tombstone option

* looks like we need to add a TombstoneEventProcessor anyway.

* ...so let's add it to the options if the SDK supports it (adapt to >= S, since that is the earliest version where `REASON_CRASH_NATIVE` provides tombstones via the `traceInputStream`)

* Adapt `AndroidEnvelopeCache` to also write Tombstone timestamp markers

* Integrate handling of historical Tombstone option + last tombstone marker

This currently does not work:

While we now can optionally enable reporting of "historical" tombstones, by making the `TombstoneHint` `Backfillable` it will automatically be enriched by the `ANRv2EventProcessor` which is currently the only `BackfillingEventProcessor`.

The `ANRv2EventProcessor` is partially written in way that is potentially generic for events with `Backfillable` hints, but other parts are enriching as if those are always were ANRs, which up to now was true, but with Tombstones that assumption now breaks.

Next Steps:

* There is considerable duplication between the ANRv2Integration and TombstoneIntegration

* sprinkle with TODOs to highlight next PR steps

* fix typo

* implement TombstoneParser as Closable and close the tombstone stream

* tighten code with final and null annotations.

* eliminate obsolete null check

* Deduplicate ApplicationExitInfo handling for corresponding Integrations via HistoryDispatcher

* also update api dumps and formatting for sentry changes

* update tombstone message construction

* fix abortMessage check

* convert AnrV2EventProcessor to a more generic ApplicationExitInfoEventProcessor

this handles all events with Backfillable hint, but adds an interface HintEnricher, to allow hint-specific enrichment (like for ANRs) before and after the generic backfilling happened.

* reintroduce, update and correct old inline docs where they make sense.

* remove obsolete TombstoneEventProcessor

* clean up tombstone error handling

* convert ApplicationExitInfoHistoryDispatcher.removeLatest() to use an iterator instead of for-each

* add TombstoneIntegrationTest

* replace assertions with explicit RuntimeException

* use test base class for AEI

* remove obsolete TODO RE: mechanism change

* clarify open TODO RE: AbnormalExit discriminator for ANR hints

* refactor instructionAddressAdjustment from String to enum

* deduplicate hex formatting of addresses and TombstoneParser

* adapt tryEndPreviousSession() in EnvelopeCache to NativeCrashExit hint

* move isBackgroundAnr() to the AnrHintEnricher since it is only used there.

* revert method-references back to lambdas

* clean up exception assignment in TombstoneParser

* properly format the debug-id as an OLE GUID

* remove obsolete equality between code-id and debug-id

* ensure that errors in the char->digit conversion is also handled

* fix debugId/codeId look-up and checks in TombstoneIntegrationTest

* make movePreviousSession idempotent wrt the previous session

* reuse existing codeId -> debugId conversion instead of adding a new converter

* make PreviousSessionFinalizer aware of tombstone handled sessions and add crashedLastRun

* bump SDK config for TombstoneIntegrationTest to version 31 (S)

* add changelog entry

* retrieve test tombstone.pb fixture as a resource rather than a file path.

* attribute pb to be binary and re-commit the non-corrupted tombstone.pb

* reset crashedLastRun in PreviousSessionFinalizer and add test-case

* format

* fix Ok Session termination test

* format

* clarify comments and variable names in PreviousSessionFinalizer

* only enable distribution and sizeAnalysis if the SENTRY_AUTH_TOKEN environment variable is present and `isNotBlank()``

* compress tombstone test resource to reduce its repo weight down to less than 20%

* add round-trip serialization test for `instructionAddressAdjustment`

---------

Co-authored-by: Markus Hintersteiner <[email protected]>

Author: supervacuus

.gitattributes

@@ -1,6 +1,8 @@
 * text eol=lf
 *.png binary
 *.jpg binary
+*.pb binary
+*.gz binary
 
 # These are explicitly windows files and should use crlf
 *.bat           text eol=crlf

CHANGELOG.md

@@ -15,6 +15,12 @@
 - Discard envelopes on `4xx` and `5xx` response ([#4950](https://github.com/getsentry/sentry-java/pull/4950))
   - This aims to not overwhelm Sentry after an outage or load shedding (including HTTP 429) where too many events are sent at once
 
+### Feature
+
+- Add a Tombstone integration that detects native crashes without relying on the NDK integration, but instead using `ApplicationExitInfo.REASON_CRASH_NATIVE` on Android 12+. ([#4933](https://github.com/getsentry/sentry-java/pull/4933))
+  - Currently exposed via options as an _internal_ API only.
+  - If enabled alongside the NDK integration, crashes will be reported as two separate events. Users should enable only one; deduplication between both integrations will be added in a future release.
+
 ## 8.29.0
 
 ### Fixes

gradle/libs.versions.toml

@@ -41,6 +41,7 @@ spotless = "7.0.4"
 gummyBears = "0.12.0"
 camerax = "1.3.0"
 openfeature = "1.18.2"
+protobuf = "4.33.1"
 
 [plugins]
 kotlin-android = { id = "org.jetbrains.kotlin.android", version.ref = "kotlin" }
@@ -60,6 +61,7 @@ spotless = { id = "com.diffplug.spotless", version.ref = "spotless" }
 detekt = { id = "io.gitlab.arturbosch.detekt", version = "1.23.8" }
 jacoco-android = { id = "com.mxalbert.gradle.jacoco-android", version = "0.2.0" }
 kover = { id = "org.jetbrains.kotlinx.kover", version = "0.7.3" }
+protobuf = { id = "com.google.protobuf", version = "0.9.5" }
 vanniktech-maven-publish = { id = "com.vanniktech.maven.publish", version = "0.30.0" }
 springboot2 = { id = "org.springframework.boot", version.ref = "springboot2" }
 springboot3 = { id = "org.springframework.boot", version.ref = "springboot3" }
@@ -139,6 +141,8 @@ otel-javaagent-extension-api = { module = "io.opentelemetry.javaagent:openteleme
 otel-semconv = { module = "io.opentelemetry.semconv:opentelemetry-semconv", version.ref = "otelSemanticConventions" }
 otel-semconv-incubating = { module = "io.opentelemetry.semconv:opentelemetry-semconv-incubating", version.ref = "otelSemanticConventionsAlpha" }
 p6spy = { module = "p6spy:p6spy", version = "3.9.1" }
+protobuf-javalite = { module = "com.google.protobuf:protobuf-javalite", version.ref = "protobuf"}
+protoc = { module = "com.google.protobuf:protoc", version.ref = "protobuf" }
 quartz = { module = "org.quartz-scheduler:quartz", version = "2.3.0" }
 reactor-core = { module = "io.projectreactor:reactor-core", version = "3.5.3" }
 retrofit = { module = "com.squareup.retrofit2:retrofit", version.ref = "retrofit" }

sentry-android-core/api/sentry-android-core.api

@@ -141,13 +141,6 @@ public final class io/sentry/android/core/AnrIntegrationFactory {
 	public static fun create (Landroid/content/Context;Lio/sentry/android/core/BuildInfoProvider;)Lio/sentry/Integration;
 }
 
-public final class io/sentry/android/core/AnrV2EventProcessor : io/sentry/BackfillingEventProcessor {
-	public fun <init> (Landroid/content/Context;Lio/sentry/android/core/SentryAndroidOptions;Lio/sentry/android/core/BuildInfoProvider;)V
-	public fun getOrder ()Ljava/lang/Long;
-	public fun process (Lio/sentry/SentryEvent;Lio/sentry/Hint;)Lio/sentry/SentryEvent;
-	public fun process (Lio/sentry/protocol/SentryTransaction;Lio/sentry/Hint;)Lio/sentry/protocol/SentryTransaction;
-}
-
 public class io/sentry/android/core/AnrV2Integration : io/sentry/Integration, java/io/Closeable {
 	public fun <init> (Landroid/content/Context;)V
 	public fun close ()V
@@ -203,6 +196,13 @@ public final class io/sentry/android/core/AppState$LifecycleObserver : androidx/
 	public fun onStop (Landroidx/lifecycle/LifecycleOwner;)V
 }
 
+public final class io/sentry/android/core/ApplicationExitInfoEventProcessor : io/sentry/BackfillingEventProcessor {
+	public fun <init> (Landroid/content/Context;Lio/sentry/android/core/SentryAndroidOptions;Lio/sentry/android/core/BuildInfoProvider;)V
+	public fun getOrder ()Ljava/lang/Long;
+	public fun process (Lio/sentry/SentryEvent;Lio/sentry/Hint;)Lio/sentry/SentryEvent;
+	public fun process (Lio/sentry/protocol/SentryTransaction;Lio/sentry/Hint;)Lio/sentry/protocol/SentryTransaction;
+}
+
 public final class io/sentry/android/core/BuildConfig {
 	public static final field BUILD_TYPE Ljava/lang/String;
 	public static final field DEBUG Z
@@ -352,6 +352,8 @@ public final class io/sentry/android/core/SentryAndroidOptions : io/sentry/Sentr
 	public fun isEnableSystemEventBreadcrumbs ()Z
 	public fun isEnableSystemEventBreadcrumbsExtras ()Z
 	public fun isReportHistoricalAnrs ()Z
+	public fun isReportHistoricalTombstones ()Z
+	public fun isTombstoneEnabled ()Z
 	public fun setAnrEnabled (Z)V
 	public fun setAnrReportInDebug (Z)V
 	public fun setAnrTimeoutIntervalMillis (J)V
@@ -381,6 +383,8 @@ public final class io/sentry/android/core/SentryAndroidOptions : io/sentry/Sentr
 	public fun setNativeHandlerStrategy (Lio/sentry/android/core/NdkHandlerStrategy;)V
 	public fun setNativeSdkName (Ljava/lang/String;)V
 	public fun setReportHistoricalAnrs (Z)V
+	public fun setReportHistoricalTombstones (Z)V
+	public fun setTombstoneEnabled (Z)V
 }
 
 public abstract interface class io/sentry/android/core/SentryAndroidOptions$BeforeCaptureCallback {
@@ -469,6 +473,29 @@ public final class io/sentry/android/core/SystemEventsBreadcrumbsIntegration : i
 	public fun register (Lio/sentry/IScopes;Lio/sentry/SentryOptions;)V
 }
 
+public class io/sentry/android/core/TombstoneIntegration : io/sentry/Integration, java/io/Closeable {
+	public fun <init> (Landroid/content/Context;)V
+	public fun close ()V
+	public fun register (Lio/sentry/IScopes;Lio/sentry/SentryOptions;)V
+}
+
+public final class io/sentry/android/core/TombstoneIntegration$TombstoneHint : io/sentry/hints/BlockingFlushHint, io/sentry/hints/Backfillable, io/sentry/hints/NativeCrashExit {
+	public fun <init> (JLio/sentry/ILogger;JZ)V
+	public fun isFlushable (Lio/sentry/protocol/SentryId;)Z
+	public fun setFlushable (Lio/sentry/protocol/SentryId;)V
+	public fun shouldEnrich ()Z
+	public fun timestamp ()Ljava/lang/Long;
+}
+
+public class io/sentry/android/core/TombstoneIntegration$TombstonePolicy : io/sentry/android/core/ApplicationExitInfoHistoryDispatcher$ApplicationExitInfoPolicy {
+	public fun <init> (Lio/sentry/android/core/SentryAndroidOptions;)V
+	public fun buildReport (Landroid/app/ApplicationExitInfo;Z)Lio/sentry/android/core/ApplicationExitInfoHistoryDispatcher$Report;
+	public fun getLabel ()Ljava/lang/String;
+	public fun getLastReportedTimestamp ()Ljava/lang/Long;
+	public fun getTargetReason ()I
+	public fun shouldReportHistorical ()Z
+}
+
 public final class io/sentry/android/core/UserInteractionIntegration : android/app/Application$ActivityLifecycleCallbacks, io/sentry/Integration, java/io/Closeable {
 	public fun <init> (Landroid/app/Application;Lio/sentry/util/LoadClass;)V
 	public fun close ()V
@@ -495,11 +522,15 @@ public final class io/sentry/android/core/ViewHierarchyEventProcessor : io/sentr
 }
 
 public final class io/sentry/android/core/cache/AndroidEnvelopeCache : io/sentry/cache/EnvelopeCache {
+	public static final field LAST_ANR_MARKER_LABEL Ljava/lang/String;
 	public static final field LAST_ANR_REPORT Ljava/lang/String;
+	public static final field LAST_TOMBSTONE_MARKER_LABEL Ljava/lang/String;
+	public static final field LAST_TOMBSTONE_REPORT Ljava/lang/String;
 	public fun <init> (Lio/sentry/android/core/SentryAndroidOptions;)V
 	public fun getDirectory ()Ljava/io/File;
 	public static fun hasStartupCrashMarker (Lio/sentry/SentryOptions;)Z
 	public static fun lastReportedAnr (Lio/sentry/SentryOptions;)Ljava/lang/Long;
+	public static fun lastReportedTombstone (Lio/sentry/SentryOptions;)Ljava/lang/Long;
 	public fun store (Lio/sentry/SentryEnvelope;Lio/sentry/Hint;)V
 	public fun storeEnvelope (Lio/sentry/SentryEnvelope;Lio/sentry/Hint;)Z
 }

sentry-android-core/build.gradle.kts

@@ -8,6 +8,7 @@ plugins {
   alias(libs.plugins.jacoco.android)
   alias(libs.plugins.errorprone)
   alias(libs.plugins.gradle.versions)
+  alias(libs.plugins.protobuf)
 }
 
 android {
@@ -83,6 +84,7 @@ dependencies {
   implementation(libs.androidx.lifecycle.common.java8)
   implementation(libs.androidx.lifecycle.process)
   implementation(libs.androidx.core)
+  implementation(libs.protobuf.javalite)
 
   errorprone(libs.errorprone.core)
   errorprone(libs.nopen.checker)
@@ -109,3 +111,10 @@ dependencies {
   testRuntimeOnly(libs.androidx.fragment.ktx)
   testRuntimeOnly(libs.timber)
 }
+
+protobuf {
+  protoc { artifact = libs.protoc.get().toString() }
+  generateProtoTasks {
+    all().forEach { task -> task.builtins { create("java") { option("lite") } } }
+  }
+}

sentry-android-core/src/main/java/io/sentry/android/core/AndroidOptionsInitializer.java

@@ -5,6 +5,7 @@
 import android.app.Application;
 import android.content.Context;
 import android.content.pm.PackageInfo;
+import android.os.Build;
 import io.sentry.CompositePerformanceCollector;
 import io.sentry.DeduplicateMultithreadedEventProcessor;
 import io.sentry.DefaultCompositePerformanceCollector;
@@ -188,7 +189,8 @@ static void initializeIntegrationsAndProcessors(
     options.addEventProcessor(new PerformanceAndroidEventProcessor(options, activityFramesTracker));
     options.addEventProcessor(new ScreenshotEventProcessor(options, buildInfoProvider));
     options.addEventProcessor(new ViewHierarchyEventProcessor(options));
-    options.addEventProcessor(new AnrV2EventProcessor(context, options, buildInfoProvider));
+    options.addEventProcessor(
+        new ApplicationExitInfoEventProcessor(context, options, buildInfoProvider));
     if (options.getTransportGate() instanceof NoOpTransportGate) {
       options.setTransportGate(new AndroidTransportGate(options));
     }
@@ -373,6 +375,10 @@ static void installDefaultIntegrations(
     final Class<?> sentryNdkClass = loadClass.loadClass(SENTRY_NDK_CLASS_NAME, options.getLogger());
     options.addIntegration(new NdkIntegration(sentryNdkClass));
 
+    if (buildInfoProvider.getSdkInfoVersion() >= Build.VERSION_CODES.S) {
+      options.addIntegration(new TombstoneIntegration(context));
+    }
+
     // this integration uses android.os.FileObserver, we can't move to sentry
     // before creating a pure java impl.
     options.addIntegration(EnvelopeFileObserverIntegration.getOutboxFileObserver());