Handle Browsertrix's now-normalized URLs when importing WARCs

[Mr0grog]

In v1.11.2, Browsertrix started normalizing seed URLs, so it no longer necessarily requests the actual configured URLs from the seed file. That means we can no longer just look for the seeds in the resulting WARCs when we want to extract page data, the request/response records in the WARCs will sometimes use URLs that are not semantically equivalent to the seeds.

For more on this, see:

• bug(normalize): normalize urls for seeds, address array-valued headers webrecorder/browsertrix-crawler#959
• Seed URL normalization in v1.11.2 changes what URLs are fetched and recorded webrecorder/browsertrix-crawler#982

We already do some normalization, but only that which is strictly equivalent in a URL (e.g. lower-casing hostnames). This adds a second function for making “matchable” URLs that is used instead of the normalized URL in places where we need to find WARC records that match seed URLs. (Crucially, this is only used for matching — when we import data into other systems, we always use the actual URL from the WARC record.)

Right now matchable_url() does pretty minimal changes (just sorts the query parameters), but we may want to make it more aggressive (at the extreme end, we could just use SURT). At current, we’d have to change this again if Browsertrix made its normalizations more aggressive.

[Mr0grog]

NOTE: another refactor I’d had sitting around came along for the ride here. It’s just renaming a couple models and variables to be a little clearer about what they represent.