-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
feat(replay): Mobile Replay performance overhead #11989
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 15 commits
4059ca2
3a53bf0
735fcf7
7c8c7e3
6252573
1d6646a
9b324fd
5e34e6c
5cde0be
272ba00
ea752e5
2e8acfb
ea4d452
5ba12f9
3ca4d9d
e676a5f
337e600
17f19ea
9b9453c
f0de54d
e8fc586
2b8441b
01be474
c634e6e
f96d1ab
a0ec705
b24170c
c6e8e3e
06dfb69
3360c7d
19be384
800e00d
c6d5210
b7eab55
e11cb9c
77d80e6
47fe5fd
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| --- | ||
| title: Performance Overhead | ||
| sidebar_order: 5502 | ||
| notSupported: | ||
| description: "Learn more about how enabling Session Replay impacts the performance of your application." | ||
| --- | ||
|
|
||
| The only way to get accurate metrics is to measure performance overhead yourself. You can apply realistic access patterns against your own application, and correlate that to your own topline business metrics. You can learn more about the various optimizations implemented in the Android Replay SDK in the [Replay Performance Overhead](/product/explore/session-replay/mobile/performance-overhead/) documentation. | ||
|
|
||
| We measured the overhead of the Android Replay SDK using the [Pocket Casts](https://github.com/Automattic/pocket-casts-android) open-source app, which includes a diverse set of components such as Fragments, Activities, and Jetpack Compose screens. The benchmarks were conducted with full masking enabled, using optimized release builds, and running through the same user flow 10 times. This approach ensured the measurements closely reflect the overhead in a real-world application. Here are the results (median values are shown): | ||
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| | Metric | Sentry SDK only | Sentry + Replay SDK | | ||
| | -------------------------------- | --------------- | ------------------- | | ||
| | FPS | 55 fps | 54 fps | | ||
| | Memory | 255 MB | 265 MB | | ||
| | CPU | 36% | 42% | | ||
| | App Startup Time (Cold) | 1533.35 ms | 1539.55 ms | | ||
| | Main Thread Time | - | 20ms | | ||
| | Network Bandwidth | - | 35 KB | | ||
|
|
||
|
|
||
| The benchmarks were run on a Pixel 2XL with Android 11. | ||
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| <Note> | ||
|
|
||
| Jetpack Compose view hierarchies may be slower to snapshot initially due to ART optimizations compared to the traditional Android View System, but their performance improves as execution progresses. | ||
|
|
||
| </Note> | ||
romtsn marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| --- | ||
| title: Performance Overhead | ||
| sidebar_order: 5502 | ||
| notSupported: | ||
| description: "Learn more about how enabling Session Replay impacts the performance of your application." | ||
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| --- | ||
|
|
||
| The only way to get accurate metrics is to measure performance overhead yourself. You can apply realistic access patterns against your own application, and correlate that to your own topline business metrics. You can learn more about the various performance overhead optimizations implemented in the iOS Replay SDK in the [Replay Performance Overhead](/product/explore/session-replay/mobile/performance-overhead/) documentation. | ||
|
|
||
| We measured the overhead of the iOS Replay SDK using the [Pocket Casts](https://github.com/Automattic/pocket-casts-ios) open-source app, which includes a diverse set of components such as ViewControllers and SwiftUI screens. The benchmarks were conducted with full masking enabled, using optimized release builds, and running through the same user flow 10 times. This approach ensured the measurements closely reflect the overhead in a real-world application. Here are the results (median values are shown): | ||
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| | Metric | Sentry SDK only | Sentry + Replay SDK | | ||
| | -------------------------------- | --------------- | ------------------- | | ||
| | FPS | 55 fps | 53 fps | | ||
| | Memory | 102 MB | 121 MB | | ||
| | CPU | 4% | 13% | | ||
| | App Startup Time (Cold) | 1264.80 ms | 1265 ms | | ||
| | Main Thread Time | - | 43ms | | ||
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| | Network Bandwidth | - | 50 KB | | ||
|
|
||
|
|
||
| The benchmarks were run on an iPhone 14 Pro. | ||
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| --- | ||
| title: Performance Overhead | ||
| sidebar_order: 5502 | ||
| notSupported: | ||
| description: "Learn more about how enabling Session Replay impacts the performance of your application." | ||
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| --- | ||
|
|
||
| The only way to get accurate metrics is to measure performance overhead yourself. You can apply realistic access patterns against your own application, and correlate that to your own topline business metrics. You can learn more about the various optimizations implemented in the React Native Replay SDK in the [Replay Performance Overhead](/product/explore/session-replay/mobile/performance-overhead/) documentation. | ||
|
|
||
| The React Native Replay SDK is a lightweight wrapper around the native iOS and Android SDKs, so its overhead is negligible compared to the native SDKs, making it not worth a separate measurement. Below are the benchmark results for each respective platform. | ||
|
|
||
| We measured the overhead using the [Pocket Casts iOS](https://github.com/Automattic/pocket-casts-ios) and [Pocket Casts Android](https://github.com/Automattic/pocket-casts-android) open-source apps respectively, which include a diverse set of components such as ViewControllers and SwiftUI screens. The benchmarks were conducted with full masking enabled, using optimized release builds, and running through the same user flow 10 times. This approach ensured the measurements closely reflect the overhead in a real-world application. Here are the results (median values are shown): | ||
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ## iOS | ||
|
|
||
| | Metric | Sentry SDK only | Sentry + Replay SDK | | ||
| | -------------------------------- | --------------- | ------------------- | | ||
| | FPS | 55 fps | 53 fps | | ||
| | Memory | 102 MB | 121 MB | | ||
| | CPU | 4% | 13% | | ||
| | App Startup Time (Cold) | 1264.80 ms | 1265 ms | | ||
| | Main Thread Time | - | 43ms | | ||
| | Network Bandwidth | - | 50 KB | | ||
|
|
||
|
|
||
| The benchmarks were run on an iPhone 14 Pro. | ||
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ## Android | ||
|
|
||
| | Metric | Sentry SDK only | Sentry + Replay SDK | | ||
| | -------------------------------- | --------------- | ------------------- | | ||
| | FPS | 55 fps | 54 fps | | ||
| | Memory | 255 MB | 265 MB | | ||
| | CPU | 36% | 42% | | ||
| | App Startup Time (Cold) | 1533.35 ms | 1539.55 ms | | ||
| | Main Thread Time | - | 20ms | | ||
| | Network Bandwidth | - | 35 KB | | ||
|
|
||
|
|
||
| The benchmarks were run on a Pixel 2XL with Android 11. | ||
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| --- | ||
| title: "Replay Performance Overhead" | ||
| sidebar_order: 30 | ||
| description: "Learn more about how enabling Session Replay impacts the performance of your application." | ||
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| --- | ||
|
|
||
| Session Replay for mobile works by taking snapshots of the view hierarchy as well as a screenshot within the same frame, at the rate of one per second. These screenshots are then compressed into a video file representing a segment of the user’s session. All these small segments are then streamed into Sentry with additional trace identifiers, breadcrumbs, and other useful debugging information to compose the full user session. | ||
|
|
||
| In order to perform this work without negatively impacting the host application's performance, the mobile SDKs take care to introduce minimal additional CPU and memory overhead, record screenshots and view hierarchy in a non-intrusive way and only when necessary, send the absolute minimum number of bytes required, and use low-latency ingestion endpoints geographically close to your end-users. | ||
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| **For most mobile applications, the performance overhead of our mobile SDKs will be imperceptible to end-users.** | ||
|
|
||
| ## How Is Session Replay Optimized? | ||
|
|
||
| Sentry's Session Replay SDK takes several measures to avoid negatively impacting the performance of the screen on which it's running: | ||
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| - The Session Replay SDK works by capturing screenshots, encoding them into small video segments and transmitting the content to Sentry’s servers. To reduce the amount of data transferred, multiple optimization strategies are employed: | ||
| - The SDK captures at most one screenshot per second at a 1:1 pixel density (commonly referred to as `@1x` on iOS and `mdpi` on Android). This significantly speeds up the screenshot capture process while maintaining acceptable quality. | ||
| - Screenshots are compressed into lossy `jpeg` files to reduce disk space usage and enable replay recovery in the event of a crash or an ANR. | ||
|
||
| - When sending a replay segment, the screenshots are further compressed into an `mp4` video file using `h.264` compression with a 75 kbps bitrate (the default behavior). This ensures a small payload size while preserving the replay video quality. We observed nearly a 5x reduction in size when compressing five screenshots into a video segment. | ||
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - The video segment, along with other debugging information (such as breadcrumbs), is gzip-compressed on the client before transmission over HTTP. This compression is performed on a background thread, ensuring the application's main thread remains unaffected. | ||
| - To minimize disk space usage, screenshots and video segment files are deleted from the disk upon successful upload. | ||
| - Sentry’s event ingestion infrastructure uses [distributed Points-of-Presence (PoPs)](https://blog.sentry.io/2022/03/31/sentry-points-of-presence-how-we-built-a-distributed-ingestion/) which place ingestion servers around the world and close to your users. When a Session Replay event is transmitted, the user’s device connects and transmits the event payload to the closest PoP available for their region. This greatly reduces end-to-end latency and minimizes the amount of networking overhead placed on the device. | ||
| - The SDK is built to gracefully downgrade when needed. In the event of sending too many replay events and hitting Sentry's rate limit, the SDK will disable recording to make sure the rate limit is respected. Once the rate limit is lifted, the SDK will continue recording. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can we specify the rate limit? this comes up a lot
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think we can specify it as it depends on the project/org replay quota |
||
|
|
||
| ## Factors in Performance Overhead | ||
|
|
||
| While the performance overhead of our mobile replay SDKs will be imperceptible to end-users most of the time, it can vary based on the complexity of your application. If an application has screens with a deeply-nested view tree, there will be higher overhead compared to simpler (e.g., Settings) screens. | ||
|
|
||
| If masking is enabled (the default behavior), network overhead is further reduced because the screenshots have fewer pixels, resulting in smaller file sizes. | ||
|
|
||
| There are different stages in the lifecycle of a mobile application, each with different performance characteristics. Here is a list of stages in the lifecycle, and the most important performance metrics to be aware of. | ||
|
|
||
| ### App Start | ||
|
|
||
| There is virtually no overhead from the SDK during application start up. It performs only a few lightweight tasks, such as registering the necessary listeners for system events and scheduling a background thread for capturing. | ||
|
|
||
| ### Main Thread | ||
|
|
||
| The SDK performs only the bare minimum on the main thread to capture a new frame for the replay (a screenshot and a view hierarchy snapshot) before immediately switching to a background thread for post-processing tasks. These tasks include masking screenshots, storing them in the disk cache, or encoding a video segment. This process occurs at most once per second, and it can be even less frequent depending on the frame rate setting. | ||
|
|
||
| The SDK also monitors view tree changes, and if no changes have occurred since the last frame capture, it reuses the previously captured data, completely avoiding any work on the main thread. | ||
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ### Capturing Touch Events | ||
|
|
||
| - Click/touch, swipe, and scroll events are incredibly small and fast to log; overhead is negligible. | ||
| - We debounce touch events and use smoothing for move events to balance the detail you see while debugging, with the amount of redundant data captured. | ||
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ## Measuring Performance Overhead | ||
|
|
||
| The only way to get accurate metrics is to measure performance overhead yourself. You can apply realistic access patterns against your own application, and correlate that to your own topline business metrics. | ||
romtsn marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| We measured the overhead of the Replay SDK on various open-source apps that have millions of users in production (i.e., real-world apps that have a big set of different components and features), such as [Bluesky](https://github.com/bluesky-social/social-app) or [Pocket Casts](https://github.com/Automattic/pocket-casts-android). You can check the results on the respective SDK's docs: | ||
romtsn marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| - [iOS](/platforms/apple/guides/ios/session-replay/performance-overhead/) | ||
| - [Android](/platforms/android/session-replay/performance-overhead/) | ||
| - [React Native](/platforms/react-native/session-replay/performance-overhead/) | ||
Uh oh!
There was an error while loading. Please reload this page.