Fjall instead of SurrealKV?

[MirroredLens]

Why not choose to use fjall instead of another lsm tree based database. It beats rocksdb in many benchmarks, is written in rust, and is coming into a version 3 that'll probably be even faster.

https://github.com/fjall-rs/fjall

[tobiemh]

Hi @MirroredLens thanks for the question! We actually have a benchmark tool which compares RocksDB, SurrealKV, and Fjall.

The new version of SurrealKV (from 0.10.0 up to the current 0.16.0) has been completely re-written, and is based around a combination of B+trees and LSM trees, with optimisations from many of the latest research and research papers.

The benchmarking tool crud-bench has had a minor contribution from one of the Fjall maintainers too. But we are always happy to have additional improvements if you suggest some! We would love to know how we can improve the benchmark, and which types of additional operations you would want to see!

We are working on improving scan performance at the moment and should have more releases in early 2026 as we move towards SurrealKV 1.0 🚀 !

I've just run a very quick benchmark with 15,000,000 records (roughly 2.5KiB per record, so approximately 36GiB of data), and here is a preview of the results:

Operation	Fjall	RocksDB	SurrealKV	SurrealKV vs RocksDB	Fjall vs RocksDB
CRUD Operations 5,000,000 entries
Create	142K ops/s	327K ops/s ⭐	215K ops/s	-34%	-57%
Read	923K ops/s	776K ops/s	3.1M ops/s ⭐	+306% 🚀	+19%
Update	141K ops/s	291K ops/s ⭐	209K ops/s	-28%	-52%
Delete	147K ops/s	572K ops/s ⭐	491K ops/s	-14%	-74%
Scan Operations Over 5,000,000 entries
Count All	14.9 ops/s ⭐	2.0 ops/s	2.5 ops/s	+22%	+645% 🚀
Limit (ID)	260K ops/s	519K ops/s ⭐	96K ops/s	-81%	-50%
Limit (All)	248K ops/s	523K ops/s ⭐	90K ops/s	-83%	-53%
Limit+Start (ID)	18K ops/s	41K ops/s ⭐	23K ops/s	-43%	-56%
Limit+Start (All)	13K ops/s	40K ops/s ⭐	14K ops/s	-65%	-67%
Batch Operations Batches of 100 items
Batch Create	332 ops/s	4,928 ops/s ⭐	3,887 ops/s	-21%	-93%
Batch Read	10,068 ops/s	10,380 ops/s ⭐	9,746 ops/s	-6%	-3%
Batch Update	1,229 ops/s	5,400 ops/s ⭐	4,499 ops/s	-17%	-77%
Batch Delete	6,550 ops/s	9,389 ops/s ⭐	9,355 ops/s	-0.4%	-30%
Batch Operations Batches of 1000 items
Batch Create	58 ops/s	313 ops/s	475 ops/s ⭐	+52% 🚀	-81%
Batch Read	1,958 ops/s	11,881 ops/s ⭐	9,330 ops/s	-21%	-84%
Batch Update	124 ops/s	348 ops/s	528 ops/s ⭐	+52% 🚀	-64%
Batch Delete	765 ops/s	969 ops/s	2,532 ops/s ⭐	+161% 🚀	-21%

Formatted result output can be seen here:

• result-fjall.html
• result-rocksdb.html
• result-surrealkv.html

Full result outputs can be seen here:

• result-fjall.csv
• result-rocksdb.csv
• result-surrealkv.csv

[marvin-j97]

What I do see in the benchmark results is that SKV uses much more memory than Fjall (4x) and RocksDB (2x). So maybe that causes (more) cache thrashing for them compared to SKV, and implies that the caches are configured incorrectly, or the memory measurement is not accurate.

[arriqaaq]

Hey @marvin-j97

Thank you for looking into the codebase. If you could suggest the optimal settings to use for fjall in the crud-bench, especially when comparing it against RocksDB, that would be great.

I’m currently working on several improvements, there are a number of performance issues right now. Looking forward to resolving them in the near future. As you mentioned, the requirements for SurrealKV are different from those of fjall. But fjall is great, and I’d love to see and learn where improvements can be made.

[guycipher]

It depends what Surreals goals are for their engine. They will design it how they see fit for their product. I’d recommend using some more memory and cpu, we are in 2026, fix the thrashing, care for latency and throughput, your users will thank you.

[guycipher]

I just wanna add you guys should bench amplification, latency, CV, it’s important. How do you know the engine won’t stall? Or have high latency fluctuations, it’s a science, it’s an art.

[arriqaaq]

I agree @guycipher. Noted, makes sense to add those benches

[MirroredLens]

@tobiemh thanks for the benchmarks, but could you show how the databases were built and configured.

I can't speak for every db, but fjall and rocksdb have a lot of configurations that play into the kind of workloads they're being tested for in order to most effectively use the db engine or speed up performance or make specific tradeoffs.

I'm not against SurrealKV, in fact I was inherently rooting for it back when it was trying to use adaptive radix trees. But I found tons of benefits from lsm based engines as I'm sure you guys did too. One of the core aspects of what makes them so powerful imo, is that it is highly configurable. After reading a few of fjall's blog posts, the db seemed pretty optimized and tunable, it also helps [for configs and mods] that the lsm tree itself is a seperate crate from the db engine. If the new SurrealKV is better at doing the same thing, I, as a free user have nothing but excitement. But I'm not fully convinced fjall is being used to it's fullest in your benchmarks.

Maybe the author of fjall can give more insight into this. @marvin-j97

[tobiemh]

Hi @MirroredLens you can see the benchmark code here:

• https://github.com/surrealdb/crud-bench/blob/main/src/fjall.rs
• https://github.com/surrealdb/crud-bench/blob/main/src/surrealkv.rs
• https://github.com/surrealdb/crud-bench/blob/main/src/rocksdb.rs

I used the run.sh script to run the benchmarks with 15m random entries.

Do feel free to make any improvements to the code and open a pull request.

As you said, there are configuration options for Fjall, and for SurrealKV (block size, key-value separation, cache), and a significant number of options for RocksDB of course. We have tried to choose the optimum options for the benchmark being run, and the instance specifications.

It must be noted that we love what Fjall is doing, and anything which attempts to tackle RocksDB complexity whilst written in Rust is great.

It's also important to say that SurrealKV actually aims to solve a different problem, in addition to being a performant transactional key-value store which can replace RocksDB and written in Rust - and that is to offer versioned temporal access over the key-value entries. So time becomes a third dimension which can be used to query the keys and key ranges.

Performance is obviously really important too, and with every minor release we are improving the transaction commit performance, and the range scan performance.

As you noted, LSM trees have their benefits, and one of those is being able to sync different levels to object storage in the background when each level is compacted. This is something we will be adding to SurrealKV very soon too.

[MirroredLens]

Thanks for the details, still not totally convinced it's a fair benchmark but I will say I'm excited to see adaptive radix trees in use here, hope you can get the most out of that for all it's usefulness!

[guycipher]

@tobiemh I'd be very curious how you're going to handle the time aspect. Sure you can use timestamps instead of sequence numbers, but then keeping many values of a key within the lsm tree comes at an extreme cost for space efficiency. You'd need to utilize an aggressive and adaptive compaction strategy as opposed to just leveled. Leveled will kill ya! Take a look at @nivdayan work from University of Toronto.

[arriqaaq]

Hello @guycipher! Awesome work at TidesDB, and love your work.

Firstly, thank you for going through the code and providing us feedback. I totally agree with you that leveled compaction is a bad choice for versioned data. We've just built this in the last 6-7 months, so thought of starting with a basic compaction level which would just work and then improve on it. And thanks for all the links from surrealdb/surrealkv#309

Will keep you updated on the research regarding this. And happy to take suggestions!

[guycipher]

@arriqaaq thank you and no problem at all. Can’t wait to see how SurrealKV evolves.

[marvin-j97]

@MirroredLens There are a couple of factors that could have an impact here:

• the workload is extremely multi-threaded (48): Multi-threaded writes in Fjall 2 are currently mutex'ed instead of having a proper pipeline, so concurrent writes can be much slower
• block sizes are quite high, which may hurt I/O for Fjall 2 (fixed in Fjall 3)
• the benchmark uses single-writer transactions, not optimistic transactions as it does in the other DBs

Though I find it a bit suspicious, that surrealkv, in the CRUD operations, is somehow 4.5x times faster than RocksDB while using essentially the same architecture down to the individual blocks. But on the other hand it is much slower for the scan operations. Additionally, the benchmark doesn't seem to really consider space amp, write amp, or memory usage. There are some other things in the surrealkv crate that seem a bit strange, but I won't go into detail here.
Ultimately though surrealkv seems to not support or plan to support features that Fjall has or will have in the future

[MirroredLens]

Have you considered art trees similar to https://github.com/declanvk/blart for fjall.
And when 3.0 😎

[marvin-j97]

There is SuRF which uses LOUDS encoded tries in LSM-trees, but even that is not really state-of-the-art anymore.