Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
4059ca2
feat(replay): Mobile Replay performance overhead
romtsn Nov 28, 2024
3a53bf0
Merge branch 'master' into rz/feat/session-replay-perf-overehead
romtsn Nov 28, 2024
735fcf7
Fix links
romtsn Nov 28, 2024
7c8c7e3
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
6252573
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn Nov 29, 2024
1d6646a
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
9b324fd
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
5e34e6c
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
5cde0be
Update docs/platforms/apple/guides/ios/session-replay/performance-ove…
romtsn Nov 29, 2024
272ba00
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
ea752e5
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
2e8acfb
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
ea4d452
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
5ba12f9
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
3ca4d9d
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Nov 29, 2024
e676a5f
Address PR feedback
romtsn Nov 29, 2024
337e600
Address PR feedback
romtsn Nov 29, 2024
17f19ea
Update docs/platforms/android/session-replay/performance-overhead.mdx
romtsn Dec 4, 2024
9b9453c
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Dec 4, 2024
f0de54d
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Dec 4, 2024
e8fc586
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Dec 4, 2024
2b8441b
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Dec 4, 2024
01be474
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Dec 4, 2024
c634e6e
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Dec 4, 2024
f96d1ab
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Dec 4, 2024
a0ec705
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn Dec 4, 2024
b24170c
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn Dec 4, 2024
c6e8e3e
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn Dec 4, 2024
06dfb69
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn Dec 4, 2024
3360c7d
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn Dec 4, 2024
19be384
Update docs/platforms/apple/guides/ios/session-replay/performance-ove…
romtsn Dec 4, 2024
800e00d
Update docs/platforms/apple/guides/ios/session-replay/performance-ove…
romtsn Dec 4, 2024
c6d5210
Update docs/platforms/apple/guides/ios/session-replay/performance-ove…
romtsn Dec 4, 2024
b7eab55
Update docs/platforms/android/session-replay/performance-overhead.mdx
romtsn Dec 4, 2024
e11cb9c
Update docs/platforms/android/session-replay/performance-overhead.mdx
romtsn Dec 4, 2024
77d80e6
Update docs/platforms/android/session-replay/performance-overhead.mdx
romtsn Dec 4, 2024
47fe5fd
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn Dec 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions docs/platforms/android/session-replay/performance-overhead.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
title: Performance Overhead
sidebar_order: 5502
notSupported:
description: "Learn about how enabling Session Replay impacts the performance of your application."
---

If you're considering enabling Session Replay, it's important to first understand the potential performance impact to your app. While accurate metrics require realistic testing where you apply typical access patterns and correlate the results with your business metrics, to provide a baseline, we measured the overhead using the open-source [Pocket Casts](https://github.com/Automattic/pocket-casts-android) app.

You can learn more about the various optimizations implemented in the Android Replay SDK in the [Replay Performance Overhead](/product/explore/session-replay/mobile/performance-overhead/) docs.

## Benchmarking the Android Replay SDK

The Pocket Casts app offers a diverse mix of components, including Fragments, Activities, and Jetpack Compose screens, making it an ideal candidate for testing. Here's how the benchmarks were conducted:

- **Configuration:** Full masking was enabled, and optimized release builds were used.
- **User Flow:** The same flow was executed 10 times to ensure consistency.
- **Real-World Representation:** This approach closely mirrors performance in real-world scenarios.

The benchmarks were run on a Pixel 2XL with Android 11.

### Results
Below are the results of the benchmarking tests, presented as median values to reflect typical overhead.

| Metric | Sentry SDK only | Sentry + Replay SDK |
| -------------------------------- | --------------- | ------------------- |
| FPS | 55 fps | 54 fps |
| Memory | 255 MB | 265 MB |
| CPU | 36% | 42% |
| App Startup Time (Cold) | 1533.35 ms | 1539.55 ms |
| Main Thread Time | n/a | 20ms |
| Network Bandwidth | n/a | 35 KB |



<Note>

Jetpack Compose view hierarchies may be slower to snapshot initially due to ART optimizations, compared to the traditional Android View System. But their performance improves as execution progresses.

</Note>
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
---
title: Performance Overhead
sidebar_order: 5502
notSupported:
description: "Learn about how enabling Session Replay impacts the performance of your application."
---

If you're considering enabling Session Replay, it's important to first understand the potential performance impact to your app. While accurate metrics require realistic testing where you apply typical access patterns and correlate the results with your business metrics, to provide a baseline, we measured the overhead using the open-source [Pocket Casts](https://github.com/Automattic/pocket-casts-ios) app.

You can learn more about the various performance overhead optimizations implemented in the iOS Replay SDK in the [Replay Performance Overhead](/product/explore/session-replay/mobile/performance-overhead/) docs.

## Benchmarking the iOS Replay SDK

The Pocket Casts app offers a diverse mix of components, including Fragments, Activities, and Jetpack Compose screens, making it an ideal candidate for testing. Here's how the benchmarks were conducted:
- **Configuration:** Full masking was enabled, and optimized release builds were used.
- **User Flow:** The same flow was executed 10 times to ensure consistency.
- **Real-World Representation:** This approach closely mirrors performance in real-world scenarios.

The benchmarks were run on an iPhone 14 Pro.

### Results
Below are the results of the benchmarking tests, presented as median values to reflect typical overhead.


| Metric | Sentry SDK only | Sentry + Replay SDK |
| -------------------------------- | --------------- | ------------------- |
| FPS | 55 fps | 53 fps |
| Memory | 102 MB | 121 MB |
| CPU | 4% | 13% |
| App Startup Time (Cold) | 1264.80 ms | 1265 ms |
| Main Thread Time | n/a | 43ms |
| Network Bandwidth | n/a | 50 KB |


Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
---
title: Performance Overhead
sidebar_order: 5502
notSupported:
description: "Learn about how enabling Session Replay impacts the performance of your application."
---

If you're considering enabling Session Replay, it's important to first understand the potential performance impact to your app. While accurate metrics require realistic testing where you apply typical access patterns and correlate the results with your business metrics, to provide a baseline, we measured the overhead using the open-source Pocket Casts app.

You can learn more about the various optimizations implemented in the React Native Replay SDK in the [Replay Performance Overhead](/product/explore/session-replay/mobile/performance-overhead/) docs.

The React Native Replay SDK is a lightweight wrapper for the native iOS and Android SDKs, so its overhead is minimal and doesn't require separate measurement. Below are the benchmark results for each platform.

We measured the overhead using the [Pocket Casts iOS](https://github.com/Automattic/pocket-casts-ios) and [Pocket Casts Android](https://github.com/Automattic/pocket-casts-android) open-source apps. Each includes a diverse set of components such as ViewControllers and SwiftUI screens.

Here's how the benchmarks were conducted:

- **Configuration:** Full masking was enabled, and optimized release builds were used.
- **User Flow:** The same flow was executed 10 times to ensure consistency.
- **Real-World Representation:** This approach closely mirrors performance in real-world scenarios.

The benchmarks were run on an iPhone 14 Pro and a Pixel 2XL.

## iOS

| Metric | Sentry SDK only | Sentry + Replay SDK |
| -------------------------------- | --------------- | ------------------- |
| FPS | 55 fps | 53 fps |
| Memory | 102 MB | 121 MB |
| CPU | 4% | 13% |
| App Startup Time (Cold) | 1264.80 ms | 1265 ms |
| Main Thread Time | n/a | 43ms |
| Network Bandwidth | n/a | 50 KB |



## Android

| Metric | Sentry SDK only | Sentry + Replay SDK |
| -------------------------------- | --------------- | ------------------- |
| FPS | 55 fps | 54 fps |
| Memory | 255 MB | 265 MB |
| CPU | 36% | 42% |
| App Startup Time (Cold) | 1533.35 ms | 1539.55 ms |
| Main Thread Time | n/a | 20ms |
| Network Bandwidth | n/a | 35 KB |


Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
---
title: "Replay Performance Overhead"
sidebar_order: 30
description: "Learn about how enabling Session Replay impacts the performance of your application."
---

Session Replay for mobile captures snapshots of the view hierarchy as well as a screenshot within the same frame once per second. These are compressed into a video file representing parts of the user’s session, then streamed to Sentry with trace identifiers, breadcrumbs, and other debugging information to reconstruct the full session.

To avoid impacting the host's app performance, Sentry minimizes CPU and memory usage by recording screenshots and view hierarchy non-intrusively and only when necessary, sending only essential bytes, and using low-latency ingestion endpoints geographically close to your end-users.

**For most mobile applications, the performance overhead of our Session Replay SDK will be imperceptible to end-users.**

## How Is Session Replay Optimized?

Sentry's Session Replay SDK takes several measures to avoid negatively impacting the performance of the screen on which it's running.

- The Session Replay SDK works by capturing screenshots, encoding them into small video segments and transmitting the content to Sentry’s servers. To reduce the amount of data transferred, multiple optimization strategies are employed:
- The SDK captures at most one screenshot per second at a 1:1 pixel density (commonly referred to as `@1x` on iOS and `mdpi` on Android). This significantly speeds up the screenshot capture process while maintaining acceptable quality.
- Screenshots are compressed into lossy `jpeg` files to reduce disk space usage and enable recovering of the last frames of a replay in the event of a crash or an ANR.
- When sending a replay segment, the screenshots are further compressed into an `mp4` video file using `h.264` compression with a 75 kbps bitrate (the default behavior). This ensures a small payload size while preserving video quality. We observed nearly 5x reduction in size when compressing five screenshots into a video segment.
- The video segment, along with other debugging information (such as breadcrumbs), is gzip-compressed on the client before transmission over HTTP. This compression is performed on a background thread, ensuring the application's main thread remains unaffected.
- To minimize disk space usage, screenshots and video segment files are deleted from the disk upon successful upload.
- Sentry’s event ingestion infrastructure uses [distributed Points-of-Presence (PoPs)](https://blog.sentry.io/2022/03/31/sentry-points-of-presence-how-we-built-a-distributed-ingestion/) which place ingestion servers around the world and close to your users. When a Session Replay event is transmitted, the user’s device connects and transmits the event payload to the closest PoP available for their region. This greatly reduces end-to-end latency and minimizes the amount of networking overhead placed on the device.
- The SDK is built to gracefully downgrade when needed. In the event of sending too many replay events and hitting Sentry's rate limit, the SDK will disable recording to make sure the rate limit is respected. Once the rate limit is lifted, the SDK will continue recording.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we specify the rate limit? this comes up a lot

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we can specify it as it depends on the project/org replay quota


## Factors in Performance Overhead

While the performance overhead of our Session Replay SDK will be imperceptible to end-users most of the time, it can vary based on the complexity of your application. If an application has screens with a deeply-nested view tree, there will be higher overhead compared to simpler screens (for example, Settings).

If the default masking behavior is enabled, network overhead will be further reduced because the screenshots have fewer pixels, resulting in smaller file sizes.

There are different stages in the lifecycle of a mobile application. Each has different performance characteristics. Below is a list of lifecycle stages and the most important performance metrics you should be aware of.

### App Start

There is virtually no overhead from the SDK during application start up. It performs only a few lightweight tasks, such as registering the necessary listeners for system events and scheduling a background thread for capturing.

### Main Thread

The SDK performs only the bare minimum on the main thread to capture a new frame for the replay (a screenshot and a view hierarchy snapshot) before immediately switching to a background thread for post-processing tasks. These tasks include masking screenshots, storing them in the disk cache, or encoding a video segment. This process occurs at most once per second, and it can be even less frequent depending on the frame rate setting.

The SDK also monitors view tree changes. If no changes have occurred since the last frame capture, it reuses the previously captured data, completely avoiding any work on the main thread.

### Capturing Touch Events

- Click/touch, swipe, and scroll events are incredibly small and fast to log, making the overhead negligible.
- We optimize touch event recording by debouncing (limiting the frequency of events) and smoothing move events. This ensures a clear, detailed debugging experience without capturing excessive or redundant data.

## Measuring Performance Overhead

The only way to get accurate metrics is to measure performance overhead yourself. You can apply realistic access patterns against your own application, and correlate that to your own topline business metrics.

We measured the overhead of the Replay SDK on various open-source apps that have millions of users in production. For example, real-world apps that have a large set of different components and features such as [Bluesky](https://github.com/bluesky-social/social-app) and [Pocket Casts](https://github.com/Automattic/pocket-casts-android). You can check the results by going to the SDK docs below:

- [iOS](/platforms/apple/guides/ios/session-replay/performance-overhead/)
- [Android](/platforms/android/session-replay/performance-overhead/)
- [React Native](/platforms/react-native/session-replay/performance-overhead/)
Loading