Skip to content

Conversation

@tiran
Copy link
Collaborator

@tiran tiran commented Sep 26, 2025

Redesign the resolver cache to address multiple issues.

  1. The cache now holds all candidates from the provider. Before it only stored candidates that also fulfilled the requirement and constraints.
  2. The cache now takes sdist url, GitHub repo, and GitLab project into account. Before a cache for local PyPI and pypi.org were mixed.
  3. GenericProvider no longer caches candidates. There is no way to construct a good cache key. Bootstrapper is using GenericProvider in way that does not benefit from caching either.
  4. There is just one global cache object for all providers. The new design makes it easier to clear all caches or just the cache for a single identifier.
  5. The custom logic for each provider class is now in find_candidates function. The method just has to return an iterable of candidates. The caching logic is handled by the rest of the code.
  6. Consumers can now opt out of caching with use_resolver_cache=False argument.
  7. All resolver classes now require keyword arguments. The base provider no longer takes PyPI-only arguments like include_sdists.

Fixes: #766

@tiran tiran requested a review from a team as a code owner September 26, 2025 09:10
@tiran tiran requested a review from dhellmann September 26, 2025 09:11
@mergify mergify bot added the ci label Sep 26, 2025
@tiran
Copy link
Collaborator Author

tiran commented Sep 26, 2025

test_provider_cache is expected to fail for now.

@tiran tiran force-pushed the resolver-cache branch 2 times, most recently from 06cd5ce to 2bc01ea Compare September 28, 2025 08:20
@dhellmann
Copy link
Member

It looks like ruff is not happy with the formatting of a file

Would reformat: tests/test_resolver.py

Redesign the resolver cache to address multiple issues.

1. The cache now holds all candidates from the provider. Before it only
   stored candidates that also fulfilled the requirement and
   constraints.
2. The cache now takes sdist url, GitHub repo, and GitLab project into
   account. Before a cache for local PyPI and pypi.org were mixed.
3. GenericProvider no longer caches candidates. There is no way to
   construct a good cache key. Bootstrapper is using GenericProvider in
   way that does not benefit from caching either.
4. There is just one global cache object for all providers. The new
   design makes it easier to clear all caches or just the cache for a
   single identifier.
5. The custom logic for each provider class is now in `find_candidates`
   function. The method just has to return an iterable of candidates.
   The caching logic is handled by the rest of the code.
6. Consumers can now opt out of caching with `use_resolver_cache=False`
   argument.
7. All resolver classes now require keyword arguments. The base provider
   no longer takes PyPI-only arguments like `include_sdists`.

Fixes: python-wheel-build#766
Signed-off-by: Christian Heimes <[email protected]>
@tiran tiran added enhancement New feature or request awaiting review labels Oct 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

resolver caching is overly aggressive

2 participants