feat(sdk): Implement persistent storage for ThreadEventCache#6317
Draft
Hywan wants to merge 22 commits intomatrix-org:mainfrom
Draft
feat(sdk): Implement persistent storage for ThreadEventCache#6317Hywan wants to merge 22 commits intomatrix-org:mainfrom
ThreadEventCache#6317Hywan wants to merge 22 commits intomatrix-org:mainfrom
Conversation
…adEventCacheState`. This patch checks if the `EventCacheStoreLock` is dirty when constructing `LockedThreadEventCacheState`. This patch is the first step, but it involved a bit of refactoring since `new` is now fallible and async. It propagates up to the public API which were already fallible and async fortunately for us: no breaking change. This patch changes `get_or_reload` to not clone the room ID, the store etc. every time, but rather only when the `ThreadEventCache` must be created.
This patch moves the `load_linked_chunk_metadata` function from `event_cache::caches::room::state` to `event_cache::persistence`. It's going to be useful for `ThreadEventCache`.
This patch loads the LinkedChunk metadata, in addition to lazy load the last chunk of the `ThreadEventCache`.
…ded. This patch updates the `reload` method on the `ThreadEventCacheStateLockWriteGuard` to shrink to the last chunk from the store.
…he`. This patch updates `ThreadEventCacheStateLockWriteGuard::remove_events` to also remove events from the store (it was only removing events from the in-memory LinkedChunk prior to this patch).
This patch simply inlines a variable definition to simplify the code.
This patch replaces `ThreadEventCacheStateLockWriteGuard::filter_duplicate_events` by `deduplicator::filter_duplicate_events`. First off, this function is shared between multiple event caches. Second, it deduplicates events regarding in-memory and in-store state. The calls to `ThreadEventCacheStateLockWriteGuard::remove_events` have been updated to pass the duplicated in-store events.
…rd::handle_sync.
Hywan
force-pushed
the
feat-sdk-event-cache-thread-persistent-storage
branch
from
68340ee to
5a8b3af
Compare
This patch forces to use of `use self::s` instead of `use s` to make it clear it comes from `self`. It clarifies the code here.
This patch makes `RoomEventCacheInner::weak_room`, `state`, `pagination_batch_token_notifier`, and `shared_pagination_status` private.
This patch creates the `ThreadEventCacheUpdateSender` à la `RoomEventCacheUpdateSender` to abstract the sender. This patch uses `ThreadEventCacheUpdateSender` in `ThreadEventCacheInner` (new) along with `ThreadEventCacheState`. The use in the `ThreadEventCacheInner` type is necessary to be able to send updates without reaching the state (which is behind an async faillible lock). This is necessary for pagination, see next commit.
This patch adds the ability to paginate in the store for `ThreadPagination`.
…EventCacheStateLockWriteGuard`. This patch passes the `prev_batch_token` up to `ThreadEventCacheStateLockWriteGuard::handle_sync`.
Hywan
force-pushed
the
feat-sdk-event-cache-thread-persistent-storage
branch
2 times, most recently
from
1527d82 to
c7692a4
Compare
This patch stops erasing threads when a gap is encountered. Instead, it will insert gaps as we do for rooms.
This patch renames the `test_thread_can_paginate_evn_if_seen_sync_event` to `test_thread_contains_its_root_event`. It is what it tests.
This patch replaces the construction of an event by its equivalent in a variable. The test uses `first_event` and `second_event`, but builds the “`second_event`” manually for no reason, it's just confusing
…dated. This patch fixes an invalid behaviour in a thread: when an in-thread event was redacted, it was removed from the thread. This is inconsistent regarding in-room event redaction where a redacted event is updated to its redacted form and the redaction event is added to the timeline. This patch makes the behaviour consistent by updating the redacted event. Basically, it replaces `remove_if_present` (deleted) by `replace_event_if_present` (new).
Hywan
force-pushed
the
feat-sdk-event-cache-thread-persistent-storage
branch
from
942609f to
12a1ef0
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #6317 +/- ##
==========================================
- Coverage 89.81% 89.74% -0.07%
==========================================
Files 376 377 +1
Lines 102614 102793 +179
Branches 102614 102793 +179
==========================================
+ Hits 92163 92254 +91
- Misses 6871 6959 +88
Partials 3580 3580 ☔ View full report in Codecov by Sentry. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This patch implement persistent storage for
ThreadEventCache.This contribution deserves a lot more tests but it will increase the diff size by a lot. The current strategy is to ensure all existing tests are passing. A test with a custom build of multiverse and Element X confirm it works as expected. Another PR will add all the missing tests, along with modifying the test suite of the Event Cache to share many tests between the
RoomEventCacheand theThreadEventCache.This contribution makes it work. However, refactoring is now necessary as similar code exists between
RoomEventCacheandThreadEventCache. That's for a next patch.Best to review one patch at a time. I tried to keep patch as small as possible.
CHANGELOG.mdfiles.