Merge branch 'exp/pypi-simplify' into exp/pypi-simplify-cleanup

Author: aignas

CHANGELOG.md

@@ -67,6 +67,12 @@ END_UNRELEASED_TEMPLATE
     * 3.12.11
     * 3.14.0b3
 * (toolchain) Python 3.13 now references 3.13.5
+* (gazelle) Switched back to smacker/go-tree-sitter, fixing
+  [#2630](https://github.com/bazel-contrib/rules_python/issues/2630)
+* (pypi) From now on the list of default platforms only includes `linux_x86_64`, `linux_aarch64`,
+  `osx_x86_64`, `osx_aarch64` and `windows_x86_64`. If you are on other platforms, you need to
+  use the `pip.default` to configure it yourself. If you are interested in graduating the
+  platform, consider helping set us up CI for them and update the documentation.
 
 {#v0-0-0-fixed}
 ### Fixed
@@ -84,12 +90,20 @@ END_UNRELEASED_TEMPLATE
   ({gh-issue}`3043`).
 * (pypi) The pipstar `defaults` configuration now supports any custom platform
   name.
+* (pypi) The selection of the whls has been changed and should no longer result
+  in ambiguous select matches ({gh-issue}`2759`) and should be much more efficient
+  when running `bazel query` due to fewer repositories being included 
+  ({gh-issue}`2849`).
 
 {#v0-0-0-added}
 ### Added
 * (pypi) To configure the environment for `requirements.txt` evaluation, use the newly added
   developer preview of the `pip.default` tag class. Only `rules_python` and root modules can use
   this feature. You can also configure custom `config_settings` using `pip.default`.
+* (pypi) PyPI dependencies now expose an `:extracted_whl_files` filegroup target
+  of all the files extracted from the wheel. This can be used in lieu of
+  {obj}`whl_filegroup` to avoid copying/extracting wheel multiple times to
+  get a subset of their files.
 * (gazelle) New directive `gazelle:python_generate_pyi_deps`; when `true`,
   dependencies added to satisfy type-only imports (`if TYPE_CHECKING`) and type
   stub packages are added to `pyi_deps` instead of `deps`.

CONTRIBUTING.md

@@ -234,11 +234,13 @@ merged:
 ## Binary artifacts
 
 Checking in binary artifacts is not allowed. This is because they are extremely
-problematic to verify and ensure they're safe
+problematic to verify and ensure they're safe. This is true even in
+test contexts.
 
 Examples include, but aren't limited to: prebuilt binaries, shared libraries,
 zip files, or wheels.
 
+See the dev guide for utilities to help with testing.
 
 (breaking-changes)=
 ## Breaking Changes

MODULE.bazel

@@ -74,20 +74,18 @@ pip = use_extension("//python/extensions:pip.bzl", "pip")
         env = {"platform_version": "0"},
         os_name = "linux",
         platform = "linux_{}".format(cpu),
-        platform_tags = [
+        whl_abi_tags = [
+            "abi3",
+            "cp{major}{minor}",
+        ],
+        whl_platform_tags = [
             "linux_*_{}".format(cpu),
             "manylinux_*_{}".format(cpu),
         ],
     )
     for cpu in [
         "x86_64",
         "aarch64",
-        # TODO @aignas 2025-05-19: only leave tier 0-1 cpus when stabilizing the
-        # `pip.default` extension. i.e. drop the below values - users will have to
-        # define themselves if they need them.
-        "arm",
-        "ppc",
-        "s390x",
     ]
 ]
 
@@ -103,7 +101,11 @@ pip = use_extension("//python/extensions:pip.bzl", "pip")
         env = {"platform_version": "14.0"},
         os_name = "osx",
         platform = "osx_{}".format(cpu),
-        platform_tags = [
+        whl_abi_tags = [
+            "abi3",
+            "cp{major}{minor}",
+        ],
+        whl_platform_tags = [
             "macosx_*_{}".format(suffix)
             for suffix in platform_tag_cpus
         ],
@@ -120,17 +122,27 @@ pip = use_extension("//python/extensions:pip.bzl", "pip")
     }.items()
 ]
 
-pip.default(
-    arch_name = "x86_64",
-    config_settings = [
-        "@platforms//cpu:x86_64",
-        "@platforms//os:windows",
-    ],
-    env = {"platform_version": "0"},
-    os_name = "windows",
-    platform = "windows_x86_64",
-    platform_tags = ["win_amd64"],
-)
+[
+    pip.default(
+        arch_name = cpu,
+        config_settings = [
+            "@platforms//cpu:{}".format(cpu),
+            "@platforms//os:windows",
+        ],
+        env = {"platform_version": "0"},
+        os_name = "windows",
+        platform = "windows_{}".format(cpu),
+        whl_abi_tags = [
+            "abi3",
+            "cp{major}{minor}",
+        ],
+        whl_platform_tags = whl_platform_tags,
+    )
+    for cpu, whl_platform_tags in {
+        "x86_64": ["win_amd64"],
+    }.items()
+]
+
 pip.parse(
     # NOTE @aignas 2024-10-26: We have an integration test that depends on us
     # being able to build sdists for this hub, so explicitly set this to False.
@@ -175,6 +187,8 @@ internal_dev_deps = use_extension(
 use_repo(
     internal_dev_deps,
     "buildkite_config",
+    "implicit_namespace_ns_sub1",
+    "implicit_namespace_ns_sub2",
     "rules_python_runtime_env_tc_info",
     "somepkg_with_build_files",
     "whl_with_build_files",
@@ -228,19 +242,22 @@ dev_pip = use_extension(
     dev_dependency = True,
 )
 dev_pip.parse(
+    download_only = True,
     experimental_index_url = "https://pypi.org/simple",
     hub_name = "dev_pip",
     parallel_download = False,
     python_version = "3.11",
     requirements_lock = "//docs:requirements.txt",
 )
 dev_pip.parse(
+    download_only = True,
     experimental_index_url = "https://pypi.org/simple",
     hub_name = "dev_pip",
     python_version = "3.13",
     requirements_lock = "//docs:requirements.txt",
 )
 dev_pip.parse(
+    download_only = True,
     experimental_index_url = "https://pypi.org/simple",
     hub_name = "pypiserver",
     python_version = "3.11",

docs/devguide.md

@@ -37,6 +37,12 @@ to be perfectly factored and not every common thing a test does needs to be
 factored into a more generally reusable piece. Copying and pasting is fine. It's
 more important for tests to balance understandability and maintainability.
 
+### Test utilities
+
+General code to support testing is in {gh-path}`tests/support`. It has a variety
+of functions, constants, rules etc, to make testing easier. Below are some
+common utilities that are frequently used.
+
 ### sh_py_run_test
 
 The {gh-path}`sh_py_run_test <tests/support/sh_py_run_test.bzl` rule is a helper to
@@ -68,6 +74,12 @@ the rule. To have it support setting a new flag:
   list, then modify the transition's logic to check the attribute and set
   the flag value if the attribute is set.
 
+### whl_from_dir_repo
+
+The `whl_from_dir_repo` repository rule in {gh-path}`tests/support/whl_from_dir`
+takes a directory tree and turns it into a `.whl` file. This can be used to
+create arbitrary whl files to verify functionality.
+
 ### Integration tests
 
 An integration test is one that runs a separate Bazel instance inside the test.

docs/pypi/use.md

@@ -40,9 +40,16 @@ Note that the hub repo contains the following targets for each package:
 * `@pypi//numpy:data` - the {obj}`filegroup` for all of the extra files that are included
   as data in the `pkg` target.
 * `@pypi//numpy:dist_info` - the {obj}`filegroup` for all of the files in the `<pkg prefix with version>.distinfo` directory.
+* `@pypi//numpy:extracted_whl_files` - a {obj}`filegroup` of all the files
+  extracted from the whl file.
 * `@pypi//numpy:whl` - the {obj}`filegroup` that is the `.whl` file itself, which includes all
   transitive dependencies via the {attr}`filegroup.data` attribute.
 
+:::{versionadded} VERSION_NEXT_FEATURE
+
+The `:extracted_whl_files` target was added
+:::
+
 ## Entry points
 
 If you would like to access [entry points][whl_ep], see the `py_console_script_binary` rule documentation,