feat: rework modpkg format

[Crauzer]

• Create utils functions for modpkg

[Crauzer]

btw I'm probably in favor of doing our own serialization for metadata and strings, there seem to be differences between msgpack libraries in how they handle enums for example, so I'd like to avoid potential issues there

[moonshadow565]

btw I'm probably in favor of doing our own serialization for metadata and strings, there seem to be differences between msgpack libraries in how they handle enums for example, so I'd like to avoid potential issues there

what about having metadata and strings table being just chunks instead of dedicated fields?

[Crauzer]

what about having metadata and strings table being just chunks instead of dedicated fields?

Sure, I don't think it should cause too much trouble for the user

[Crauzer]

@moonshadow565 @alanpq starting this up again

[moonshadow565]

@moonshadow565 @alanpq starting this up again

Been thinking about it, no need to overcomplicate it.
Just make the format simple as possible (maybe with binrw ?).
For signature checking we can just generate dedicated TOC + sha256 of real metadata that we sign instead.

[alanpq]

+1 on the binrw/etc usage, since we own both sides we might as well keep it as boilerplate free as possible

[alanpq]

@Crauzer @moonshadow565 few breaking changes with the binrw port:

• Modpkg's metadata, chunk_paths & wad_paths are no longer stored in the chunk map
• the strings inside of ModpkgLicense are null terminated now instead of len-prefixed (this feels better to me but can easily change it back)
• there might be more that I've either forgotten or missed, so I recommend poking around yourselves

Non-breaking but I've also added proptest for checking the soundness of size()

[moonshadow565]

There should be better way to do this with binrw

[alanpq]

As far as I know, we either do this or we use some custom String struct that impls BinWrite/Read (or we ask binrw to implement mapping inside an option somehow)

[moonshadow565]

Why not just implement dummy Write that tracks the size and write metdata into it?

See https://docs.rs/binrw/latest/binrw/io/trait.Write.html

[alanpq]

Github mobile lied to me, thought this was a comment on the optional_string_write map stuff. There probably is a better way I just ported what already was there - I'll let @Crauzer decide what to replace it with

[Crauzer]

Honestly, I don't even see the point of this function existing now that we're using binrw. I think we can remove it. If needed, we can add it back using some other way as moon said.

[moonshadow565]

Same as ModpkgMetadata::size

[Crauzer]

Going to merge this as it is, we can further refine the format as needed while I work on the mod project packer.