Introduce a new set-based function stdlib_modules()

[AA-Turner]

Follows on from #167, cc @woodruffw @miketheman

We introduce a new frozenset-based API, stdlib_modules(), that takes the place of stdlib_list for loading the package data. Due to this, we can simplify the remaining functions by using sorted() in stdlib_list and removing the lru_cache decorator.

A

[woodruffw]

Thanks @AA-Turner! Makes sense to me.

Do you have thoughts/opinions on deprecating the stdlib_list() API? I'd be okay with us marking it as deprecated and then removing it in the N+1 minor release (or maybe finally going 1.0?), but I'm curious what you think.

[AA-Turner]

Do you have thoughts/opinions on deprecating the stdlib_list() API? I'd be okay with us marking it as deprecated and then removing it in the N+1 minor release (or maybe finally going 1.0?), but I'm curious what you think.

I would support deprecating, yes. I can't think of any good reasons to require a sorted list as first-class API. sys.stdlib_module_names is a set, other similar packages also use a frozenset approach.

The outlier is sys.builtin_module_names which is a tuple(!) for reasons of seeming prehistory -- the set type didn't yet exist.

finally going 1.0

I'd also support this, version numbers are unending in their supply. If anything goes badly wrong, it would be fun to finally see a package use the PEP 440 Epoch marker!

A

[miketheman]

warehouse currently uses stdlib_list here:
https://github.com/pypi/warehouse/blob/2ecf83f9938573cc89b3e79991b934d27016018f/warehouse/packaging/services.py#L60-L73

Probably a relatively trivial replacement, but it'd be good to understand what API is better before it's removed.

[woodruffw]

warehouse currently uses stdlib_list here: pypi/warehouse@2ecf83f/warehouse/packaging/services.py#L60-L73

Probably a relatively trivial replacement, but it'd be good to understand what API is better before it's removed.

If I'm reading this right, the two should have equivalent characteristics in this case: itertools.chain should work just as well over a frozenset. But this is indeed a good reason to not do the deprecation immediately, and instead add this API and see what happens when we try and migrate PyPI to it 🙂

[AA-Turner]

@miketheman the other relevant part is here:

        if canonicalize_name(name) in STDLIB_PROHIBITED:
            raise ProjectNameUnavailableStdlibError()

This could probably be replaced by the proposed API in #120, because (I assume) all warehouse wants to check is that the proposed name does not attempt to inhabit any part of the stdlib namespace. 'collections.tin_of_spam' in STDLIB_PROHIBITED would currently be False!

Proof of concept: https://pypi.org/project/collections.tin_of_spam/