-
Notifications
You must be signed in to change notification settings - Fork 4
Open
Labels
Description
Issue
With adding of asynchronous query execution it was added one more put to query history.
Before we were just saving QueryRecord containing result, but now we also save intermediate QueryStatus::Runing
.
Saving intermediate query status enables our WebUI to see information about running queries, in a natural way just fetching a history.
But this approach impacts significantly on a query execution latency.
Proposal
Add caching layer in SlateDBHistoryStore, so:
- When put
QueryRecord
it added to the cache, then spawned async task and result returned without waiting completion of the put; - So no wait when saving items to the history;
- Cache should work transparently, so
HistoryStore
interface behave like it access items as usually. Cached data mixed with the real data loaded from slatedb in proper way; - Saver task also uses
ExecutionService::wait_historical_query_result
to be notified when to remove item from the cache.
Thoughts
Implementing this would require some effort, maybe it's not a better timing for this to come up.