Skip to content

Commit 5e667b7

Browse files
committed
First self review
1 parent 343f111 commit 5e667b7

File tree

1 file changed

+13
-15
lines changed

1 file changed

+13
-15
lines changed

docs/ex_perfomance_metrics.md

Lines changed: 13 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -7,30 +7,28 @@
77
<hr />
88

99
## Objective
10-
Crashes are already tracked on Sentry. Performance metrics are also useful health metrics to check. This can be tracked by Sentry too.
10+
Crashes are already tracked on Sentry. Performance metrics are also useful health metrics to check. They can be tracked by Sentry too.
1111

12-
This document specifies what metrics we track and how we measure them. It has a focus on EX but it could be extended (and renamed) to track any matrix client app performance.
12+
This document specifies what metrics we track and how we measure them. It focuses on EX but it could be extended (and renamed) to track any matrix client app performance.
1313

1414
## UX metrics
1515

16-
17-
| Title | Android name / Description | What it is measuring | Initial and final conditions | Tech metrics | Notes |
16+
| Title | Android name / Description | What is it measuring? | Initial and final conditions | Tech metrics | Notes |
1817
| :---- | :---- | :---- | :---- | :---- | :---- |
19-
| Cold start time | Cold start until cached room list is displayed | How long it takes from launching the app to displaying cached data on the screen | From:<ul><li>The user is already connected</li><li>The app is not running in background User taps on the app icon</li></ul> To:<ul><li>The room list is fully loaded from the permanent cached and displayed to the end user | <ul><li>First rooms displayed after login or restoration (TBD) </li></ul> | It does not consider Sentry init time and the check that the user has consented from disk. |
20-
| Catch-up | Room list says it is resumed and running | How long until the room list is up-to-date | From:<ul><li>The app was inactive in background and get debackgrounded</li><li>Or the app just finished its cold start</li></ul> To:<ul><li>The room list service turns into `Running` No more Syncing spinner | | The expected final conditions should be:<ul><li>The room list is updated</li><li>Properly sorted</li><li>Last messages are up-to-date</li></ul> But we need more work to get it |
21-
| Notification to message | A notification was tapped and it opened a timeline | How long it takes from taping the notification to the message becoming visible in the app | From:<ul><li>The app is in background or active The user taps on a notification</li></ul> To:<ul><li>The message is visible in the timeline The timeline around this message is fully loaded | | |
18+
| Cold start | Cold start until the cached room list is displayed | How long it takes from launching the app to displaying cached data on the screen | From:<ul><li>The user is already connected</li><li>The app is not running in background</li><li> The user taps on the app icon</li></ul> To:<ul><li>The room list is fully loaded from the permanent cached and displayed to the end user | <ul><li>First rooms displayed after login or restoration (**TBD**) </li></ul> | The clock starts after the Sentry is initialised. |
19+
| Catch-up | The app syncs and the room list becomes up-to-date | How long until the room list is up-to-date | From:<ul><li>The app was inactive in background and get debackgrounded</li><li>Or the app just finished its cold start</li></ul> To:<ul><li>The room list service state becomes [`Running`](https://github.com/matrix-org/matrix-rust-sdk/blob/matrix-sdk-ui-0.14.0/bindings/matrix-sdk-ffi/src/sync_service.rs#L36)</li><li>No more Syncing spinner | | The expected final conditions should be:<ul><li>The room list is updated</li><li>Properly sorted</li><li>Last messages are up-to-date</li></ul> But we need more work to compute this metric |
20+
| Notification to message | A notification was tapped and it opened a timeline | How long it takes from taping the notification to the message becoming visible in the app | From:<ul><li>The app is in background or active</li><li>The user taps on a notification</li></ul> To:<ul><li>The message is visible in the timeline</li><li>The timeline around this message is fully loaded | | |
2221
| Open a room | Open a room and see loaded items in the timeline | How long it takes from tapping a room in the room list to displaying a full page of messages | From:<ul><li>User taps a room from the room list</li></ul> To:<ul><li>The timeline is fully loaded with first items loaded | <ul><li>Get and display first timeline items</li></ul> | |
23-
| Send a message | ? | How long it takes from tapping send to the message appearing in the timeline as sent | From:<ul><li>User hits the send button</li></ul> To:<ul><li>The timeline shows it as sent | | Needs to be tested for validation |
24-
22+
| Send a message | Send to sent state in timeline | How long it takes from tapping send to the message appearing in the timeline as sent | From:<ul><li>User hits the send button</li></ul> To:<ul><li>The timeline shows it as sent | | **TODO**: We need to experiment the feasibility |
2523

2624
## Additional data
2725

2826
We need to add some data to the metrics so that we can better characterise them:
2927

30-
| Data | Why it is useful | Notes |
28+
| Data | Why is it useful? | Notes |
3129
| :---- | :---- | :---- |
32-
| Device information | <ul><li>It helps to detect specific device problem, especially on the fragmented Android ecosystem</li></ul> | It is provided by Sentry. No additional work |
33-
| Homeserver URL hash | <ul><li>To compare matrix.org and element.io homeservers speeds</li><li>Measure the impact of a slow server</li></ul> | We use SHA-512 for the hash |
34-
| DBs file size | <ul><li>Check the impact of growing DB on speed performance</li><li>Check disk space used by the app</li></ul> | We have 4 DBs: state, event, media, crypto |
35-
| Account size | <ul><li>Check the impact on speed performance</li><li>Check the impact on \`/sync\` response size</li></ul> | It should be part of a new stats FFI API |
36-
| First /sync response size | <ul><li>Check it stays as small as possible</li></ul> | It should be part of a new stats FFI API |
30+
| Device information | <ul><li>To detect specific device problem, especially on the fragmented Android ecosystem</li></ul> | It is provided by Sentry |
31+
| Homeserver URL hash | <ul><li>To compare matrix.org and element.io homeservers speed</li><li>To measure the impact of a slow homeserver</li></ul> | We use SHA-512 to compute the hash. </br>**TBD:** Define the input string (`https://matrix.org`, `matrix.org`() or `https://matrix-client.matrix.org` (`base_url`) or something else) |
32+
| DBs file size | <ul><li>To check the impact of growing DB on speed performance</li><li>To check disk space used by the app</li></ul> | We have 4 DBs: state, event, media, crypto.</br>**TODO**: Add here names from the new Rust SDK API |
33+
| Account size | <ul><li>To check the impact on speed performance</li><li>To check the impact on \`/sync\` response size</li></ul> | It is the number of rooms joined by the user.</br>**TODO**: Add the Rust SDK API for reference |
34+
| First /sync response size | <ul><li>To check network usage stays as low as possible</li></ul> | **TODO**: Add the Rust SDK API for reference |

0 commit comments

Comments
 (0)