RocksDB and TitanDB support and sync with upstream

[sykp241095]

原帖作者：harpocrates
原帖时间：2023-03-05 17:26:57.487862
原帖ID/Slug：442 / rocksdb-and-titandb-support-and-sync-with-upstream

I’ve watched a very interesting video with Sunny Brains and Andy Zhang at CNCF. Regarding what you did on RocksDB, replacing its WAL with Raft log, I suppose that Titan being a plugin on top of RocksDB it inherits it as well, right ?

Also RocksDB as a new implementation called BlobDB, do you have in mind to allow users to access BlobDB parameters (like enable_blob_* and blob_*) to allow users to switch to BlobDB instead of RocksDB ? Do you have any idea how Titan compares to BlobDB ?

Also regarding RocksDB the upstream is currently on 7.9.2 (2022-12-21) when TiKV is on 6.29 (2022-03-29 if it’s 6.29.5). What’s the current RoadMap on getting new changes/fixes from the upstream ?

[sykp241095]

原帖作者：winkyao
原帖时间：2023-03-06 02:18:42.336417

I created a topic in internals.tidb.io, this question is related to TiKV team, you can find the response there: RocksDB and TitanDB support and sync with upstream - storage - TiDB Internals

[sykp241095]

原帖作者：harpocrates
原帖时间：2023-03-09 15:13:31.350923

Hey @winkyao I created an account 2 days ago and it’s still on hold at this time. I tried to notify you there but got no response. Can you see what’s going on ? It doesn’t seem there is a lot of activity on that internal forum …

[sykp241095]

原帖作者：tabokie
原帖时间：2023-03-10 04:08:31.904218

I suppose that Titan being a plugin on top of RocksDB it inherits it as well, right ?

Yes, Titan doesn’t change the foreground write path of RocksDB, all the work we did to remove RocksDB WAL will apply to Titan as well.

do you have in mind to allow users to access BlobDB parameters (like enable_blob_* and blob_*) to allow users to switch to BlobDB instead of RocksDB ?

We are not currently seeking to find a “better” Titan. For example, let’s assume there’s a performance issue with Titan, we will try to fix it instead of replacing it with BlobDB. So as a result we won’t be putting much energy into supporting BlobDB.

Do you have any idea how Titan compares to BlobDB ?

We don’t. Titan was developed years ago, at that time BlobDB is still incomplete and unstable. (In fact, the primary author of Titan was from the BlobDB team.) Over the years our focus has drifted away from Titan because it has a smaller user base and a lower priority. So we haven’t got the time to compare it against the latest BlobDB yet.

What’s the current RoadMap on getting new changes/fixes from the upstream ?

We would very much like to sync with upstream regularly (2 months for example). But we are really short on bandwidth right now. So the current status is we internally schedule some updates (which will very likely be delayed), without promising any roadmap publicly.