Skip to content

Conversation

YaroslavLitvinov
Copy link
Contributor

Closes #1752

PR includes

  • slatedb version update to 0.8.2
    • It brings less granularity on error, by providing ErrorKind
  • Eliminate writer latency for puts coming from query history, by using less durable WriteOptions

Copy link
Contributor

SLT Targeted Testing: No relevant SLT tests found for the changes in this PR. No testing required.

@YaroslavLitvinov
Copy link
Contributor Author

Changing writer options significantly improved performance, though some query history related tests fails:

failures:
    tests::worksheets::test_ui_worksheets
    tests::worksheets::test_ui_worksheets_search
    tests::worksheets::test_ui_worksheets_sort

@JanKaul
Copy link
Contributor

JanKaul commented Sep 26, 2025

Weird, does the UI get its values from SlateDB or is it directly accessing the objectstore?
Stupid question, it has to go through SlateDB. Weird, because it should be in SlateDBs cache.

@YaroslavLitvinov
Copy link
Contributor Author

Actually tests with that option are flaky, as once there failing 2 of them, then 3. And in CI/CD none is failing

@DanCodedThis
Copy link
Contributor

Weird, does the UI get its values from SlateDB or is it directly accessing the objectstore? Stupid question, it has to go through SlateDB. Weird, because it should be in SlateDBs cache.

What about PutOptions TTL could this affect it somehow (a shot in the dark)?

@DanCodedThis
Copy link
Contributor

Also ObjectStoreCacheOptions on SlateDb creation

@DanCodedThis
Copy link
Contributor

Also ObjectStoreCacheOptions on SlateDb creation

It seems default cache options don't activate this

@DanCodedThis
Copy link
Contributor

It seems default options doesn't set up anything or at all if you check out the DbBuilder build? I think we should definitely double check all options, settings, etc for our SlateDb instance.

@DanCodedThis
Copy link
Contributor

By default they provide Moka & Foyer caches. Is Foyer worse in our case? We use Moka, however we can even build our own cache if necessary with their DbCache trait.

cc @osipovartem @rampage644 @Vedin

@DanCodedThis
Copy link
Contributor

There are also DBReaderOptions which set up their own cache (Foyer or Moka also). But I didn't see where those options are used yet.

@YaroslavLitvinov YaroslavLitvinov marked this pull request as draft September 27, 2025 00:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Draft: Decrease query latency caused by puts into query history

4 participants