Skip to content
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
4059ca2
feat(replay): Mobile Replay performance overhead
romtsn Nov 28, 2024
3a53bf0
Merge branch 'master' into rz/feat/session-replay-perf-overehead
romtsn Nov 28, 2024
735fcf7
Fix links
romtsn Nov 28, 2024
7c8c7e3
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
6252573
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn Nov 29, 2024
1d6646a
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
9b324fd
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
5e34e6c
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
5cde0be
Update docs/platforms/apple/guides/ios/session-replay/performance-ove…
romtsn Nov 29, 2024
272ba00
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
ea752e5
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
2e8acfb
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
ea4d452
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
5ba12f9
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
3ca4d9d
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
e676a5f
Address PR feedback
romtsn Nov 29, 2024
337e600
Address PR feedback
romtsn Nov 29, 2024
17f19ea
Update docs/platforms/android/session-replay/performance-overhead.mdx
romtsn Dec 4, 2024
9b9453c
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Dec 4, 2024
f0de54d
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Dec 4, 2024
e8fc586
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Dec 4, 2024
2b8441b
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Dec 4, 2024
01be474
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Dec 4, 2024
c634e6e
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Dec 4, 2024
f96d1ab
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Dec 4, 2024
a0ec705
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Dec 4, 2024
b24170c
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn Dec 4, 2024
c6e8e3e
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn Dec 4, 2024
06dfb69
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn Dec 4, 2024
3360c7d
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn Dec 4, 2024
19be384
Update docs/platforms/apple/guides/ios/session-replay/performance-ove…
romtsn Dec 4, 2024
800e00d
Update docs/platforms/apple/guides/ios/session-replay/performance-ove…
romtsn Dec 4, 2024
c6d5210
Update docs/platforms/apple/guides/ios/session-replay/performance-ove…
romtsn Dec 4, 2024
b7eab55
Update docs/platforms/android/session-replay/performance-overhead.mdx
romtsn Dec 4, 2024
e11cb9c
Update docs/platforms/android/session-replay/performance-overhead.mdx
romtsn Dec 4, 2024
77d80e6
Update docs/platforms/android/session-replay/performance-overhead.mdx
romtsn Dec 4, 2024
47fe5fd
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn Dec 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions docs/platforms/android/session-replay/performance-overhead.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
title: Performance Overhead
sidebar_order: 5502
notSupported:
description: "Learn more about how enabling Session Replay impacts the performance of your application."
---

The only way to get accurate metrics is to measure performance overhead yourself. You can apply realistic access patterns against your own application, and correlate that to your own topline business metrics. You can learn more about the various optimizations implemented in the Android Replay SDK in the [Replay Performance Overhead](/product/explore/session-replay/mobile/performance-overhead/) documentation.

We measured the overhead of the Android Replay SDK using the [Pocket Casts](https://github.com/Automattic/pocket-casts-android) open-source app, which includes a diverse set of components such as Fragments, Activities, and Jetpack Compose screens. The benchmarks were conducted with full masking enabled, using optimized release builds, and running through the same user flow 10 times. This approach ensured the measurements closely reflect the overhead in a real-world application. Here are the results (median values are shown):

| Metric | Sentry SDK only | Sentry + Replay SDK |
| -------------------------------- | --------------- | ------------------- |
| FPS | 55 fps | 54 fps |
| Memory | 255 MB | 265 MB |
| CPU | 36% | 42% |
| App Startup Time (Cold) | 1533.35 ms | 1539.55 ms |
| Main Thread Time | n/a | 20ms |
| Network Bandwidth | n/a | 35 KB |


The benchmarks were run on a Pixel 2XL with Android 11.

<Note>

Jetpack Compose view hierarchies may be slower to snapshot initially due to ART optimizations compared to the traditional Android View System, but their performance improves as execution progresses.

</Note>
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
title: Performance Overhead
sidebar_order: 5502
notSupported:
description: "Learn more about how enabling Session Replay impacts the performance of your application."
---

The only way to get accurate metrics is to measure performance overhead yourself. You can apply realistic access patterns against your own application, and correlate that to your own topline business metrics. You can learn more about the various performance overhead optimizations implemented in the iOS Replay SDK in the [Replay Performance Overhead](/product/explore/session-replay/mobile/performance-overhead/) documentation.

We measured the overhead of the iOS Replay SDK using the [Pocket Casts](https://github.com/Automattic/pocket-casts-ios) open-source app, which includes a diverse set of components such as ViewControllers and SwiftUI screens. The benchmarks were conducted with full masking enabled, using optimized release builds, and running through the same user flow 10 times. This approach ensured the measurements closely reflect the overhead in a real-world application. Here are the results (median values are shown):

| Metric | Sentry SDK only | Sentry + Replay SDK |
| -------------------------------- | --------------- | ------------------- |
| FPS | 55 fps | 53 fps |
| Memory | 102 MB | 121 MB |
| CPU | 4% | 13% |
| App Startup Time (Cold) | 1264.80 ms | 1265 ms |
| Main Thread Time | n/a | 43ms |
| Network Bandwidth | n/a | 50 KB |


The benchmarks were run on an iPhone 14 Pro.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we bring this up to the main paragraph above here? I don't think it needs to be isolated

Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
title: Performance Overhead
sidebar_order: 5502
notSupported:
description: "Learn more about how enabling Session Replay impacts the performance of your application."
---

The only way to get accurate metrics is to measure performance overhead yourself. You can apply realistic access patterns against your own application, and correlate that to your own topline business metrics. You can learn more about the various optimizations implemented in the React Native Replay SDK in the [Replay Performance Overhead](/product/explore/session-replay/mobile/performance-overhead/) documentation.

The React Native Replay SDK is a lightweight wrapper around the native iOS and Android SDKs, so its overhead is negligible compared to the native SDKs, making it not worth a separate measurement. Below are the benchmark results for each respective platform.

We measured the overhead using the [Pocket Casts iOS](https://github.com/Automattic/pocket-casts-ios) and [Pocket Casts Android](https://github.com/Automattic/pocket-casts-android) open-source apps respectively, which include a diverse set of components such as ViewControllers and SwiftUI screens. The benchmarks were conducted with full masking enabled, using optimized release builds, and running through the same user flow 10 times. This approach ensured the measurements closely reflect the overhead in a real-world application. Here are the results (median values are shown):

## iOS

| Metric | Sentry SDK only | Sentry + Replay SDK |
| -------------------------------- | --------------- | ------------------- |
| FPS | 55 fps | 53 fps |
| Memory | 102 MB | 121 MB |
| CPU | 4% | 13% |
| App Startup Time (Cold) | 1264.80 ms | 1265 ms |
| Main Thread Time | n/a | 43ms |
| Network Bandwidth | n/a | 50 KB |


The benchmarks were run on an iPhone 14 Pro.

## Android

| Metric | Sentry SDK only | Sentry + Replay SDK |
| -------------------------------- | --------------- | ------------------- |
| FPS | 55 fps | 54 fps |
| Memory | 255 MB | 265 MB |
| CPU | 36% | 42% |
| App Startup Time (Cold) | 1533.35 ms | 1539.55 ms |
| Main Thread Time | n/a | 20ms |
| Network Bandwidth | n/a | 35 KB |


The benchmarks were run on a Pixel 2XL with Android 11.
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
---
title: "Replay Performance Overhead"
sidebar_order: 30
description: "Learn more about how enabling Session Replay impacts the performance of your application."
---

Session Replay for mobile works by taking snapshots of the view hierarchy as well as a screenshot within the same frame, at the rate of one per second. These screenshots are then compressed into a video file representing a segment of the user’s session. All these small segments are then streamed into Sentry with additional trace identifiers, breadcrumbs, and other useful debugging information to compose the full user session.

In order to perform this work without negatively impacting the host application's performance, Sentry employs a number of measures to minimize additional CPU and memory overhead, record screenshots and view hierarchy in a non-intrusive way and only when necessary, send the absolute minimum number of bytes required, and use low-latency ingestion endpoints geographically close to your end-users.

**For most mobile applications, the performance overhead of our Session Replay SDK will be imperceptible to end-users.**

## How Is Session Replay Optimized?

Sentry's Session Replay SDK takes several measures to avoid negatively impacting the performance of the screen on which it's running:

- The Session Replay SDK works by capturing screenshots, encoding them into small video segments and transmitting the content to Sentry’s servers. To reduce the amount of data transferred, multiple optimization strategies are employed:
- The SDK captures at most one screenshot per second at a 1:1 pixel density (commonly referred to as `@1x` on iOS and `mdpi` on Android). This significantly speeds up the screenshot capture process while maintaining acceptable quality.
- Screenshots are compressed into lossy `jpeg` files to reduce disk space usage and enable recovering of the last frames of a replay in the event of a crash or an ANR.
- When sending a replay segment, the screenshots are further compressed into an `mp4` video file using `h.264` compression with a 75 kbps bitrate (the default behavior). This ensures a small payload size while preserving the replay video quality. We observed nearly a 5x reduction in size when compressing five screenshots into a video segment.
- The video segment, along with other debugging information (such as breadcrumbs), is gzip-compressed on the client before transmission over HTTP. This compression is performed on a background thread, ensuring the application's main thread remains unaffected.
- To minimize disk space usage, screenshots and video segment files are deleted from the disk upon successful upload.
- Sentry’s event ingestion infrastructure uses [distributed Points-of-Presence (PoPs)](https://blog.sentry.io/2022/03/31/sentry-points-of-presence-how-we-built-a-distributed-ingestion/) which place ingestion servers around the world and close to your users. When a Session Replay event is transmitted, the user’s device connects and transmits the event payload to the closest PoP available for their region. This greatly reduces end-to-end latency and minimizes the amount of networking overhead placed on the device.
- The SDK is built to gracefully downgrade when needed. In the event of sending too many replay events and hitting Sentry's rate limit, the SDK will disable recording to make sure the rate limit is respected. Once the rate limit is lifted, the SDK will continue recording.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we specify the rate limit? this comes up a lot

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we can specify it as it depends on the project/org replay quota


## Factors in Performance Overhead

While the performance overhead of our Session Replay SDK will be imperceptible to end-users most of the time, it can vary based on the complexity of your application. If an application has screens with a deeply-nested view tree, there will be higher overhead compared to simpler (e.g., Settings) screens.

If masking is enabled (the default behavior), network overhead is further reduced because the screenshots have fewer pixels, resulting in smaller file sizes.

There are different stages in the lifecycle of a mobile application, each with different performance characteristics. Here is a list of stages in the lifecycle, and the most important performance metrics to be aware of.

### App Start

There is virtually no overhead from the SDK during application start up. It performs only a few lightweight tasks, such as registering the necessary listeners for system events and scheduling a background thread for capturing.

### Main Thread

The SDK performs only the bare minimum on the main thread to capture a new frame for the replay (a screenshot and a view hierarchy snapshot) before immediately switching to a background thread for post-processing tasks. These tasks include masking screenshots, storing them in the disk cache, or encoding a video segment. This process occurs at most once per second, and it can be even less frequent depending on the frame rate setting.

The SDK also monitors view tree changes, and if no changes have occurred since the last frame capture, it reuses the previously captured data, completely avoiding any work on the main thread.

### Capturing Touch Events

- Click/touch, swipe, and scroll events are incredibly small and fast to log; overhead is negligible.
- We debounce touch events and use smoothing for move events to balance the detail you see while debugging, with the amount of redundant data captured.

## Measuring Performance Overhead

The only way to get accurate metrics is to measure performance overhead yourself. You can apply realistic access patterns against your own application, and correlate that to your own topline business metrics.

We measured the overhead of the Replay SDK on various open-source apps that have millions of users in production (i.e., real-world apps that have a big set of different components and features), such as [Bluesky](https://github.com/bluesky-social/social-app) or [Pocket Casts](https://github.com/Automattic/pocket-casts-android). You can check the results on the respective SDK's docs:

- [iOS](/platforms/apple/guides/ios/session-replay/performance-overhead/)
- [Android](/platforms/android/session-replay/performance-overhead/)
- [React Native](/platforms/react-native/session-replay/performance-overhead/)