[Help] Trying to upgrade to recent RocksDB

[sykp241095]

Original author: harpocrates
Original time: 2023-03-21 03:04:01.332989
Original ID/Slug: 477 / help-trying-to-upgrade-to-recent-rocksdb

I’m trying to upgrade RocksDB to a more recent version and I have the following error when compiling the code I have at the moment https://pastebin.com/raw/W0KaSCjJ. What is weird is that the part that doesn’t compile is exactly the same as in the current version of tikv/rocksdb and it fails on rocksdb/include/rocksdb/db.h at 6.29.tikv · tikv/rocksdb · GitHub.

What is annoying is that it complains about 2 things as you can see, a conversion issue and the fact there is some deprecated code. But the same code in your repo with the exact same compiler doesn’t fail. So any idea what is going on ? What is also surprising is that vscode sees the same conversion error in my repo but not in yours when both files are similar there. It happens after it parses some Makefile or CMakeFile but I can’t see where it comes atm. I also try to run the exact same cc1plus command and it fails the same. Any guidance would be much appreciated, @tabokie can you help on this ?

[sykp241095]

Original author: tabokie
Original time: 2023-03-22 05:03:09.329376

Hi, I am on leave this couple of weeks. But I take a look at your errors, it should be related to these PRs:

• Remove Deprecated overloads of DB::GetApproximateSizes by akankshamahajan15 · Pull Request #9458 · facebook/rocksdb · GitHub
• Change enum SizeApproximationFlags to enum class by akankshamahajan15 · Pull Request #9604 · facebook/rocksdb · GitHub

[sykp241095]

Original author: harpocrates
Original time: 2023-03-27 03:23:35.458949

Hey @tabokie sorry I just realized I didn’t respond but yeah it helped. I’m slowly making some progress and will keep asking you on this thread if I’m stuck again. Thanks

[sykp241095]

Original author: harpocrates
Original time: 2023-03-31 00:52:56.448724

Hey @tabokie, I’m still trying to make it work. Can you tell me what am I supposed to do to confirm it’s working as expected before I create a PR ? I’m running make check in a docker container, which is taking years tbh. Is there anything else I can do ? Would it be better to create a PR and have you allow it to run on your CircleCI somehow ?

[sykp241095]

Original author: tabokie
Original time: 2023-03-31 11:58:10.82254

Yeah, you can do that. Normally we just make sure the code can compile and is formatted before filing a PR.

[sykp241095]

Original author: harpocrates
Original time: 2023-03-31 21:35:41.418638

Can you create a new branch or tell me which branch I should create the PR on ? I suppose it doesn’t make sense to create it on 6.29.tikv and you have a dev branch somewhere.
I only have a few tests that fail now. It will be easier to talk about them when the PR is created

t/run-compact_files_test-CompactFilesTest.FilterContext
ALL db_kv_checksum_test . DbKvChecksumTestMergedBatch
3.151 FAIL t/run-db_write_test-DBWriteTestInstance-DBWriteTest.PostWriteCallback-0

3.141 FAIL t/run-db_write_test-DBWriteTestInstance-DBWriteTest.PostWriteCallback-2
4.371 FAIL t/run-listener_test-EventListenerTest.OnSingleDBSubcompactionTest

If I execute one of DBWriteTest.PostWriteCallback-? it succeeds one or multiple times then crashes

[sykp241095]

Original author: harpocrates
Original time: 2023-04-05 15:15:52.165128

@tabokie ^

[sykp241095]

Original author: tabokie
Original time: 2023-04-10 09:47:25.430316

@harpocrates That’s really impressive work! Thanks for putting in the time.

For the PR, you can create one against 6.29.tikv. If needed I will make some adjustments to our internal CI so that your changes can compile there. Once the changes are approved, I will manually create a branch with your changes in it as the new default branch.

And BTW sorry for the delay, I am not paying much attention to the forum. If there’s any problem you can ping me on GitHub more easily.