Build a resilient, efficient and user-friendly KV store

[zz-jason]

Goals

Currently, recovery from disk and continuous checkpoint is not fully supported, write-ahead logs may missing or incorrect, and tests are needed to guarantee the correctness. The goal of this TODO list is to make the project:

• Resilient, feel confident to be used in production
• Efficient: especially for read-mostly workloads
• User-friendly: easy to integrate, for example, as a storage extension to Redis, MongoDB, PostgreSQL, MySQL

You can join the slack workspace for discussion, any help is welcomed and appreciated!

Logging and recovering

• Recover from btree merge
• Recover from BasicKV remove/update/insert
• Remove transaction-related fields from BasicKV wal entries
• Support checkpoint

MVCC Transaction and concurrency control

• Fully support FatTuple and adaptive version storage #131
• Support system transaction #125
• Support multi-SSDs

Tests

• Increase test coverage to 80% #119
• Enable asan test

Checks

• clang-format
• Make clang-tidy happy #117 (Help-Wanted, Good-First-Issue)
• Include-What-You-Use

Usability

• Build a simple key-value REPL playground, such as Redis.

Misc

• Build with clang
• Cache workflow dependencies to speed up CI
• Is there a better way to replace setjmp/longjmp since it's error-prone, how about c++20 stackless coroutine (Coroutines (C++20), lewissbaker/cppcoro)?

[NewtonVan]

I'm also interested in this project & I also want to participate in it. But I'm a rookie in this field. I wish we could have more discussion about this.

[zz-jason]

@NewtonVan Welcome! Which part are you interested in? You can find us in the slack workspace for discussion!