Skip to content

Add EX Performance metrics doc #3055

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
manuroe merged 10 commits into from
Jan 30, 2026
Merged

Add EX Performance metrics doc #3055

Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
343f111
Migration from the draft our design doc
manuroe Dec 3, 2025
1b40d5e
First self review
manuroe Dec 3, 2025
da2e5b8
Integrate Sentry naming convention
manuroe Dec 3, 2025
092f7fa
Performance metrics: Refined existing UX metrics and data. Start Tech…
manuroe Dec 4, 2025
6df5b24
Add a separate table for metrics we want but can't add now
jmartinesp Dec 22, 2025
20ac5ce
Update last updated date in performance metrics
jmartinesp Dec 22, 2025
660f4e2
Add a suggested way to test 'Send a message'
jmartinesp Dec 22, 2025
fffd872
Apply suggestions from code review
manuroe Dec 22, 2025
b36f274
Rename ex_perfomance_metrics.md to ex_performance_metrics.md
manuroe Dec 22, 2025
f4e556f
UX metrics table review
manuroe Jan 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 80 additions & 0 deletions docs/ex_performance_metrics.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# EX Performance metrics

| Status | Last updated |
|--|--|
| In progress | December 22, 2025 |

<hr />

## Objective
Crashes are already tracked on Sentry. Performance metrics are also useful health metrics to check. They can be tracked by Sentry too.

This document specifies what metrics we track and how we measure them. It focuses on EX but it could be extended (and renamed) to track any matrix client app performance.

## Sentry semantics
As we use Sentry, we need to adopt its naming conventions for `Transaction(name: String, operation: String)` and `Span(operation: String, description: String)`.
More information can be found in the Sentry documentation about [Transaction Name](https://docs.sentry.io/platforms/native/enriching-events/transaction-name/) and [Span Operations](https://develop.sentry.dev/sdk/performance/span-operations/).

## UX metrics

They are tracked using Sentry transactions. `Transaction.operation` is `ux` to reflect high-level, user journey metrics.

| Metric (Sentry Transaction Name) | Description | Initial and final conditions | Tech metrics | Notes |
| :---- | :---- | :---- | :---- | :---- |
| Cached room list | Cold start until the cached room list is displayed | From:<ul><li>The user is already logged in</li><li>The app is not running in background</li><li> The user taps on the app icon</li></ul> To:<ul><li>The room list is fully loaded from the permanent cache and displayed to the end user | <ul><li>First rooms displayed after login or restoration (**TBD**) </li></ul> | The clock starts after the Sentry is initialised. |
| Up-to-date room list | The app syncs and the room list becomes up-to-date | From:<ul><li>The app was inactive in background and get debackgrounded</li><li>Or the app just finished its cold start</li></ul> To:<ul><li>The room list service state becomes [`Running`](https://github.com/matrix-org/matrix-rust-sdk/blob/matrix-sdk-ui-0.14.0/bindings/matrix-sdk-ffi/src/sync_service.rs#L36)</li><li>No more Syncing spinner | | The expected final conditions should be:<ul><li>The room list is updated</li><li>Properly sorted</li><li>Last messages are up-to-date</li></ul> But we need more work to compute this metric. |
| Notification to message | A notification was tapped and it opened a timeline | From:<ul><li>The app is in background or active</li><li>The notification is for a message in the main timeline</li><li>The user taps on the notification</li></ul> To:<ul><li>The message is visible in the timeline</li><li>The timeline around this message is fully loaded | <ul><li>Timeline load</li></ul> | |
| Open a room | Open a room and see loaded items in the timeline | From:<ul><li>User taps a room from the room list</li></ul> To:<ul><li>The timeline is fully loaded with first items loaded | <ul><li>Timeline load</li></ul> | |
| Send a message | Send to sent state in timeline | From:<ul><li>User hits the send button</li></ul> To:<ul><li>The timeline shows it as sent | | The `RoomSendQueueUpdate` enum in the SDK can be used for this: `NewLocalEvent` being returned means the event is in the send queue, `SentEvent` means we received it in the sync. We don't really know when it lands on the timeline, but the delay should be minimal. |

## Additional data

We need to add some data to the metrics so that we can better characterise them:

| Data | Why is it useful? | Notes |
| :---- | :---- | :---- |
| Device information | <ul><li>To detect specific device problem, especially on the fragmented Android ecosystem</li></ul> | It is provided by Sentry |
| Homeserver | <ul><li>To compare matrix.org and element.io homeservers speed</li><li>To measure the impact of a slow homeserver</li></ul> | **We use SHA-512 to compute the hash of the homeserver domain**, ie matrix.org or element.io. |
| DB files size:<ul><li>Crypto store</li><li>State store</li><li>Event cache store</li><li>Media store</li></ul> | <ul><li>To check the impact of growing DB on speed performance</li><li>To check disk space used by the app</li></ul> | Expressed in MB. |

We'd want to add this data too, but it's not possible at the moment given the info we have in the SDK:

| Data | Why is it useful? | Notes |
| :---- | :---- | :---- |
| Account size | <ul><li>To check the impact on speed performance</li><li>To check the impact on \`/sync\` response size</li></ul> | Expressed in number of joined rooms.</br>**TODO**: Add the Rust SDK API for reference<br/>**TODO**: We need to experiment the feasibility of this metric |
| Catch-up /sync size | <ul><li>To check network usage stays as low as possible</li></ul> | The response size in kB of the first `/sync` request made during catch-up.</br>**TODO**: Add the Rust SDK API for reference</br>**TODO**: We need to experiment the feasibility of this metric |

## Tech metrics

They are tracked as Sentry spans. `Span.operation` follows the [Span Operations](https://develop.sentry.dev/sdk/performance/span-operations/) convention like `http.client`. `Span.description` is the metric name.

| Metric (Sentry Span Description) | Category (Sentry Span Operation) | Description | Notes |
| :---- | :---- | :---- | :---- |
| Timeline load | `function` | `Room.timelineWithConfiguration` | |

**TODO: Add more**


## Future considerations
Later, we could add the following UX metrics.

### UX starting from outside of the app
* Call answer to call
* Permalink to message
* Notification to threaded message
* Notification to invite (room or space)

And maybe all the variants:
* Permalink to room
* Permalink to space
* Permalink to threaded message
* Permalink to profile
* Permalink to Element Call

### UX inside the app
* Make a call
* QR code login
* Open member list
* Open a permalink (of all sorts)
* Open a thread