The idea of this project

[mainrs]

I am writing a mod manager in Rust right now. Most of the modpacks I will be working with contain the same mods in the same version and configuration. Thus, I'd greatly benefit from deduplicated storage.

Is this something I can use this Library for? It seems more as that rdedup is a backup solution rather than a general purpose deduplication storage mechanism.

[dpc]

@mainrs I think so, yes. It's split into a library and binary for that very reason. You can also disable compression/encryption that you're not planning to use to save on compilation time and binary size.

[mainrs]

I suppose random access to files inside the chunks is not possible right now?
Like maybe exposing a Fuse file system on top of it. Would make my life easiert :)

[dpc]

@mainrs I vaguely remember (this crate has not been getting a lot of action in recent years and I've been busy with other things) discussions about random access. It should be possible to relatively easy to implement, by adding some byte-offset data into chunk indexes, so the whole thing turns into a deduplicated B-Tree, roughly.

I could answer some questions, but I'm so busy that I'm not going to implement myself, even though it does seem neat.

Also - I should warn you that that restore path is currently naively implemented, so not as fast as it could. Making a backup is much more common, so I only ever focused on that.