-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
feat(replay): Mobile Replay performance overhead #11989
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+179
−0
Merged
Changes from 36 commits
Commits
Show all changes
37 commits
Select commit
Hold shift + click to select a range
4059ca2
feat(replay): Mobile Replay performance overhead
romtsn 3a53bf0
Merge branch 'master' into rz/feat/session-replay-perf-overehead
romtsn 735fcf7
Fix links
romtsn 7c8c7e3
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn 6252573
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn 1d6646a
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn 9b324fd
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn 5e34e6c
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn 5cde0be
Update docs/platforms/apple/guides/ios/session-replay/performance-ove…
romtsn 272ba00
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn ea752e5
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn 2e8acfb
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn ea4d452
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn 5ba12f9
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn 3ca4d9d
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn e676a5f
Address PR feedback
romtsn 337e600
Address PR feedback
romtsn 17f19ea
Update docs/platforms/android/session-replay/performance-overhead.mdx
romtsn 9b9453c
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn f0de54d
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn e8fc586
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn 2b8441b
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn 01be474
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn c634e6e
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn f96d1ab
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn a0ec705
Update docs/product/explore/session-replay/mobile/performance-overhea…
romtsn b24170c
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn c6e8e3e
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn 06dfb69
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn 3360c7d
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn 19be384
Update docs/platforms/apple/guides/ios/session-replay/performance-ove…
romtsn 800e00d
Update docs/platforms/apple/guides/ios/session-replay/performance-ove…
romtsn c6d5210
Update docs/platforms/apple/guides/ios/session-replay/performance-ove…
romtsn b7eab55
Update docs/platforms/android/session-replay/performance-overhead.mdx
romtsn e11cb9c
Update docs/platforms/android/session-replay/performance-overhead.mdx
romtsn 77d80e6
Update docs/platforms/android/session-replay/performance-overhead.mdx
romtsn 47fe5fd
Update docs/platforms/react-native/session-replay/performance-overhea…
romtsn File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
40 changes: 40 additions & 0 deletions
40
docs/platforms/android/session-replay/performance-overhead.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| --- | ||
| title: Performance Overhead | ||
| sidebar_order: 5502 | ||
| notSupported: | ||
| description: "Learn about how enabling Session Replay impacts the performance of your application." | ||
| --- | ||
|
|
||
| If you're considering enabling Session Replay, it's important to first understand the potential performance impact to your app. While accurate metrics require realistic testing where you apply typical access patterns and correlate the results with your business metrics, to provide a baseline, we measured the overhead using the open-source [Pocket Casts](https://github.com/Automattic/pocket-casts-android) app. | ||
|
|
||
| You can learn more about the various optimizations implemented in the Android Replay SDK in the [Replay Performance Overhead](/product/explore/session-replay/mobile/performance-overhead/) docs. | ||
|
|
||
| ## Benchmarking the Android Replay SDK | ||
|
|
||
| The Pocket Casts app offers a diverse mix of components, including Fragments, Activities, and Jetpack Compose screens, making it an ideal candidate for testing. Here's how the benchmarks were conducted: | ||
|
|
||
| - **Configuration:** Full masking was enabled, and optimized release builds were used. | ||
| - **User Flow:** The same flow was executed 10 times to ensure consistency. | ||
| - **Real-World Representation:** This approach closely mirrors performance in real-world scenarios. | ||
|
|
||
| The benchmarks were run on a Pixel 2XL with Android 11. | ||
|
|
||
| ### Results | ||
| Below are the results of the benchmarking tests, presented as median values to reflect typical overhead. | ||
|
|
||
| | Metric | Sentry SDK only | Sentry + Replay SDK | | ||
| | -------------------------------- | --------------- | ------------------- | | ||
| | FPS | 55 fps | 54 fps | | ||
| | Memory | 255 MB | 265 MB | | ||
| | CPU | 36% | 42% | | ||
| | App Startup Time (Cold) | 1533.35 ms | 1539.55 ms | | ||
| | Main Thread Time | n/a | 20ms | | ||
| | Network Bandwidth | n/a | 35 KB | | ||
|
|
||
|
|
||
|
|
||
| <Note> | ||
|
|
||
| Jetpack Compose view hierarchies may be slower to snapshot initially due to ART optimizations, compared to the traditional Android View System. But their performance improves as execution progresses. | ||
|
|
||
| </Note> |
34 changes: 34 additions & 0 deletions
34
docs/platforms/apple/guides/ios/session-replay/performance-overhead.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| --- | ||
| title: Performance Overhead | ||
| sidebar_order: 5502 | ||
| notSupported: | ||
| description: "Learn about how enabling Session Replay impacts the performance of your application." | ||
| --- | ||
|
|
||
| If you're considering enabling Session Replay, it's important to first understand the potential performance impact to your app. While accurate metrics require realistic testing where you apply typical access patterns and correlate the results with your business metrics, to provide a baseline, we measured the overhead using the open-source [Pocket Casts](https://github.com/Automattic/pocket-casts-ios) app. | ||
|
|
||
| You can learn more about the various performance overhead optimizations implemented in the iOS Replay SDK in the [Replay Performance Overhead](/product/explore/session-replay/mobile/performance-overhead/) docs. | ||
|
|
||
| ## Benchmarking the iOS Replay SDK | ||
|
|
||
| The Pocket Casts app offers a diverse mix of components, including Fragments, Activities, and Jetpack Compose screens, making it an ideal candidate for testing. Here's how the benchmarks were conducted: | ||
| - **Configuration:** Full masking was enabled, and optimized release builds were used. | ||
| - **User Flow:** The same flow was executed 10 times to ensure consistency. | ||
| - **Real-World Representation:** This approach closely mirrors performance in real-world scenarios. | ||
|
|
||
| The benchmarks were run on an iPhone 14 Pro. | ||
|
|
||
| ### Results | ||
| Below are the results of the benchmarking tests, presented as median values to reflect typical overhead. | ||
|
|
||
|
|
||
| | Metric | Sentry SDK only | Sentry + Replay SDK | | ||
| | -------------------------------- | --------------- | ------------------- | | ||
| | FPS | 55 fps | 53 fps | | ||
| | Memory | 102 MB | 121 MB | | ||
| | CPU | 4% | 13% | | ||
| | App Startup Time (Cold) | 1264.80 ms | 1265 ms | | ||
| | Main Thread Time | n/a | 43ms | | ||
| | Network Bandwidth | n/a | 50 KB | | ||
|
|
||
|
|
48 changes: 48 additions & 0 deletions
48
docs/platforms/react-native/session-replay/performance-overhead.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| --- | ||
| title: Performance Overhead | ||
| sidebar_order: 5502 | ||
| notSupported: | ||
| description: "Learn about how enabling Session Replay impacts the performance of your application." | ||
| --- | ||
|
|
||
| If you're considering enabling Session Replay, it's important to first understand the potential performance impact to your app. While accurate metrics require realistic testing where you apply typical access patterns and correlate the results with your business metrics, to provide a baseline, we measured the overhead using the open-source [Pocket Casts](https://github.com/Automattic/pocket-casts-android) app. | ||
|
|
||
| You can learn more about the various optimizations implemented in the React Native Replay SDK in the [Replay Performance Overhead](/product/explore/session-replay/mobile/performance-overhead/) docs. | ||
|
|
||
| The React Native Replay SDK is a lightweight wrapper for the native iOS and Android SDKs, so its overhead is minimal and doesn't require separate measurement. Below are the benchmark results for each platform. | ||
|
|
||
| We measured the overhead using the [Pocket Casts iOS](https://github.com/Automattic/pocket-casts-ios) and [Pocket Casts Android](https://github.com/Automattic/pocket-casts-android) open-source apps. Each includes a diverse set of components such as ViewControllers and SwiftUI screens. | ||
|
|
||
| Here's how the benchmarks were conducted: | ||
|
|
||
| - **Configuration:** Full masking was enabled, and optimized release builds were used. | ||
| - **User Flow:** The same flow was executed 10 times to ensure consistency. | ||
| - **Real-World Representation:** This approach closely mirrors performance in real-world scenarios. | ||
|
|
||
| The benchmarks were run on an iPhone 14 Pro and a Pixel 2XL. | ||
|
|
||
| ## iOS | ||
|
|
||
| | Metric | Sentry SDK only | Sentry + Replay SDK | | ||
| | -------------------------------- | --------------- | ------------------- | | ||
| | FPS | 55 fps | 53 fps | | ||
| | Memory | 102 MB | 121 MB | | ||
| | CPU | 4% | 13% | | ||
| | App Startup Time (Cold) | 1264.80 ms | 1265 ms | | ||
| | Main Thread Time | n/a | 43ms | | ||
| | Network Bandwidth | n/a | 50 KB | | ||
|
|
||
|
|
||
|
|
||
| ## Android | ||
|
|
||
| | Metric | Sentry SDK only | Sentry + Replay SDK | | ||
| | -------------------------------- | --------------- | ------------------- | | ||
| | FPS | 55 fps | 54 fps | | ||
| | Memory | 255 MB | 265 MB | | ||
| | CPU | 36% | 42% | | ||
| | App Startup Time (Cold) | 1533.35 ms | 1539.55 ms | | ||
| | Main Thread Time | n/a | 20ms | | ||
| | Network Bandwidth | n/a | 35 KB | | ||
|
|
||
|
|
||
57 changes: 57 additions & 0 deletions
57
docs/product/explore/session-replay/mobile/performance-overhead.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| --- | ||
| title: "Replay Performance Overhead" | ||
| sidebar_order: 30 | ||
| description: "Learn about how enabling Session Replay impacts the performance of your application." | ||
| --- | ||
|
|
||
| Session Replay for mobile captures snapshots of the view hierarchy as well as a screenshot within the same frame once per second. These are compressed into a video file representing parts of the user’s session, then streamed to Sentry with trace identifiers, breadcrumbs, and other debugging information to reconstruct the full session. | ||
|
|
||
| To avoid impacting the host's app performance, Sentry minimizes CPU and memory usage by recording screenshots and view hierarchy non-intrusively and only when necessary, sending only essential bytes, and using low-latency ingestion endpoints geographically close to your end-users. | ||
|
|
||
| **For most mobile applications, the performance overhead of our Session Replay SDK will be imperceptible to end-users.** | ||
|
|
||
| ## How Is Session Replay Optimized? | ||
|
|
||
| Sentry's Session Replay SDK takes several measures to avoid negatively impacting the performance of the screen on which it's running. | ||
|
|
||
| - The Session Replay SDK works by capturing screenshots, encoding them into small video segments and transmitting the content to Sentry’s servers. To reduce the amount of data transferred, multiple optimization strategies are employed: | ||
| - The SDK captures at most one screenshot per second at a 1:1 pixel density (commonly referred to as `@1x` on iOS and `mdpi` on Android). This significantly speeds up the screenshot capture process while maintaining acceptable quality. | ||
| - Screenshots are compressed into lossy `jpeg` files to reduce disk space usage and enable recovering of the last frames of a replay in the event of a crash or an ANR. | ||
| - When sending a replay segment, the screenshots are further compressed into an `mp4` video file using `h.264` compression with a 75 kbps bitrate (the default behavior). This ensures a small payload size while preserving video quality. We observed nearly 5x reduction in size when compressing five screenshots into a video segment. | ||
| - The video segment, along with other debugging information (such as breadcrumbs), is gzip-compressed on the client before transmission over HTTP. This compression is performed on a background thread, ensuring the application's main thread remains unaffected. | ||
| - To minimize disk space usage, screenshots and video segment files are deleted from the disk upon successful upload. | ||
| - Sentry’s event ingestion infrastructure uses [distributed Points-of-Presence (PoPs)](https://blog.sentry.io/2022/03/31/sentry-points-of-presence-how-we-built-a-distributed-ingestion/) which place ingestion servers around the world and close to your users. When a Session Replay event is transmitted, the user’s device connects and transmits the event payload to the closest PoP available for their region. This greatly reduces end-to-end latency and minimizes the amount of networking overhead placed on the device. | ||
| - The SDK is built to gracefully downgrade when needed. In the event of sending too many replay events and hitting Sentry's rate limit, the SDK will disable recording to make sure the rate limit is respected. Once the rate limit is lifted, the SDK will continue recording. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can we specify the rate limit? this comes up a lot
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think we can specify it as it depends on the project/org replay quota |
||
|
|
||
| ## Factors in Performance Overhead | ||
|
|
||
| While the performance overhead of our Session Replay SDK will be imperceptible to end-users most of the time, it can vary based on the complexity of your application. If an application has screens with a deeply-nested view tree, there will be higher overhead compared to simpler screens (for example, Settings). | ||
|
|
||
| If the default masking behavior is enabled, network overhead will be further reduced because the screenshots have fewer pixels, resulting in smaller file sizes. | ||
|
|
||
| There are different stages in the lifecycle of a mobile application. Each has different performance characteristics. Below is a list of lifecycle stages and the most important performance metrics you should be aware of. | ||
|
|
||
| ### App Start | ||
|
|
||
| There is virtually no overhead from the SDK during application start up. It performs only a few lightweight tasks, such as registering the necessary listeners for system events and scheduling a background thread for capturing. | ||
|
|
||
| ### Main Thread | ||
|
|
||
| The SDK performs only the bare minimum on the main thread to capture a new frame for the replay (a screenshot and a view hierarchy snapshot) before immediately switching to a background thread for post-processing tasks. These tasks include masking screenshots, storing them in the disk cache, or encoding a video segment. This process occurs at most once per second, and it can be even less frequent depending on the frame rate setting. | ||
|
|
||
| The SDK also monitors view tree changes. If no changes have occurred since the last frame capture, it reuses the previously captured data, completely avoiding any work on the main thread. | ||
|
|
||
| ### Capturing Touch Events | ||
|
|
||
| - Click/touch, swipe, and scroll events are incredibly small and fast to log, making the overhead negligible. | ||
| - We optimize touch event recording by debouncing (limiting the frequency of events) and smoothing move events. This ensures a clear, detailed debugging experience without capturing excessive or redundant data. | ||
|
|
||
| ## Measuring Performance Overhead | ||
|
|
||
| The only way to get accurate metrics is to measure performance overhead yourself. You can apply realistic access patterns against your own application, and correlate that to your own topline business metrics. | ||
romtsn marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| We measured the overhead of the Replay SDK on various open-source apps that have millions of users in production. For example, real-world apps that have a large set of different components and features such as [Bluesky](https://github.com/bluesky-social/social-app) and [Pocket Casts](https://github.com/Automattic/pocket-casts-android). You can check the results by going to the SDK docs below: | ||
|
|
||
| - [iOS](/platforms/apple/guides/ios/session-replay/performance-overhead/) | ||
| - [Android](/platforms/android/session-replay/performance-overhead/) | ||
| - [React Native](/platforms/react-native/session-replay/performance-overhead/) | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.