Fix/overlapping overlays

[rx-dev]

Mega PR

Sadly, this is a mega PR but hopefully last of it's kind. It mostly is centered around overlay merging but includes code for updating to uv, enabling support for all resource types that are json and also some half baked code for lectern merging. It's been a long time coming so I'll spare extra wait for splitting PRs and just continue as is.

We'll follow up this PR w/ more for lectern support, weld script support, and updating to Python 3.14 + pydantic v2 for new beet.

Overlay Merging

Overlays produce a weird problem for data pack merging. When the game launches, it knows what pack format is active and prunes the pack for all of the irrelevant overlays. Weld emulates the base merging behavior of vanilla + it's extra rules. However, there's 1 problem: we don't know what version the player is launching into. This means, when we are merging packs, we have to figure out which overlays are relevant and then compute merging states where overlays are squashed for each pack format. Since iterating through each pack format would take a lot of time:

We can do 2 things:

1. Just force the user to choose a pack format (we choose newest) and pray that the user doesn't launch into a different version.
2. Create an algorithm that merges overlays intelligently based on overlapping format ranges.

It's highly likely that packs will have overlay ranges that mostly match (the exception of which are catch-all, incompatible version banners like in TCC). As long as we can segment our overlay ranges for each pack, we can compute only the necessary welded overlays that sit on top of existing overlays.

The Algorithm

If you wish to follow this section, take a look at the examples: basic_overlays and complex_overlays.

basic_overlays

This test has 2 packs, containing loot tables for the wither and the bat and 1 overlay each that overrides the wither. These overlays only exist between versions 61 and 61 (inclusive).

The expected output is a base pack with properly merged bat and wither loot tables, inclusion of the overlay loot tables for each (named for each pack), and a special merged overlay that contains both overlays merged into each other.

The algorithm first indexes all overlays and extracts the ranges for each one. After merging the base packs, we collect all the unique min and max supported format for each overlay, sort them, and then compute all of the segments. Then, we can iterate through each segment and find each overlay that intersects with it and collect them as segment overlays.

For each one, we bake each overlay onto it's pack (basically doing what the game does), ditch all files that aren't tracked by any of the overlays (essentially files that are only in the base pack), and welding into a merged state which will be a new overlay that sits on top of everything. After doing this for each segment, we can prune these for duplicates (if they exist, just checking hashes).

At the very end, we slot these packs ontop of the existing overlays which eventually make it out all the way.

complex_overlays

In this example, there's a lot of packs with various overlapping zones but the algorithm acts the same. It showcases the example where an overlay baked pack might need to be merged for 2 different segment ranges which requires some extra merging work.

Caveats

Unfortuntely, this adds a lot of processing, especially for packs with a ton of overlays. I think I still want to add an option to skip all of this if you choose a pre-baked pack format. Overall though, the codebase could use a bit of a clean up which will come in future PRs.