Database format that is suited to dwarfs

[ShalokShalom]

Hi ✨

I am looking for a method to compress my custom database with something like dwarfs.

It is used for a novel programming language that uses a database, instead of files.

In order to compress that effectively, I like to know how that database does ideally look like, to fit the algorithms of dwarfs.

What are the benefits, and what are the potential pitfalls?

Or is dwarfs maybe ideally suited for all of it, and there is no way I can develop this database that it doesn't fit right it perfectly 😉

[mhx]

Hi,

I don't understand your use case.

Did you read the DwarFS documentation and do you have some experience with the tools?

Why do you think that DwarFS would be a good fit for your use case?

It would be much easier if you already had an idea of what you're trying to achieve and how you think DwarFS could be beneficial. And it'd be much easier for you to take the (rather significant amount of) documentation already available and align that with what you have in mind than for me to guess how "a novel programming language that uses a database, instead of files" could relate to this project.

[ShalokShalom]

It would use the wasm binary representation.

And sorry, my understanding of the shape of these things on the lower level is limited.
I understand that certain patterns/memory layouts etc are more beneficial to be compressed by your algorithm than others.

If my descriptions are still too vague, you don't need to feel the need to overextend and answer.

[mhx]

It would use the wasm binary representation.

What is "it"? And why is "wasm binary representation" relevant?

And sorry, my understanding of the shape of these things on the lower level is limited. I understand that certain patterns/memory layouts etc are more beneficial to be compressed by your algorithm than others.

If my descriptions are still too vague, you don't need to feel the need to overextend and answer.

"still too vague"? All I have right now is:

• some new unspecified programming language
• somehow that PL uses an unspecified database "instead of" unspecified files (For what?)
• some unspecified thing uses a wasm binary representation (The PL? The DB? Something else?)
• no clue how any of the above relates to DwarFS

I mean, this is literally the definition of vague.

[ShalokShalom]

I dont know, what information is useful to you, as said.
I will just figure it out myself.

Thanks 🙂

[mhx]

The problem is: I have no clue what you're trying to do and, more importantly, how you're trying to incorporate DwarFS into what you're trying to do. OTOH, you don't seem to have an understanding of what DwarFS is/does. Now, the latter is documented — the former, presumably, is not. So either you indeed familiarize yourself with DwarFS, or you actually put some effort into describing what exactly you're trying to build and where exactly you think DwarFS would fit in. Even if I had all the time in the world to give you advice, I wouldn't even know where to start.