Memory usage and clog file size

[EtienneBruines]

We're currently having a namespace with some databases. The export command gave us files totaling about 100kB for all data in SurrealDB. There is very little in there at the moment. We are using the new/beta surrealkv storage engine without versioning enabled.

Now the interesting bits:

./clog/ contains a couple of 512MB files and the memory usage (RAM) of surrealdb is about 2x the sum of those 512MB files, roughly 3.5GB at the moment. The database has been running for a few months now.

We are just using the normal start options:

/usr/local/bin/surreal start --bind 127.0.0.1:9000 surrealkv:///var/lib/surrealdb

$ surreal version
2.4.0 for linux on x86_64

Any ideas on what's going on or on how to trim some data? clog sounds like it might be some contexctual logging.

[arriqaaq]

Hey @EtienneBruines

We've updated the surrealkv design, in the previous versions, the clog would represent the commit log entries (transaction log). Could you please try the latest alpha for the updated surrealkv design, which is closer to lsm

[EtienneBruines]

Hi @arriqaaq

Thanks for the super--quick reply! Interesting to see there's more improvements in the 3-alpha available! I just re-created the database (as the manifest-version is incompatible) and am curious to see how it evolves over the next few months. Very hopeful though 😄