Merge remote-tracking branch 'upstream/main' into remove-cc-defs

Author: rickeylev

.bazelci/presubmit.yml

@@ -177,7 +177,7 @@ tasks:
     <<: *minimum_supported_version
     <<: *reusable_config
     name: "RBE: Ubuntu, minimum Bazel"
-    platform: rbe_ubuntu1604
+    platform: rbe_ubuntu2004
     build_flags:
       # BazelCI sets --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1,
       # which prevents cc toolchain autodetection from working correctly
@@ -195,7 +195,7 @@ tasks:
   rbe:
     <<: *reusable_config
     name: "RBE: Ubuntu"
-    platform: rbe_ubuntu1604
+    platform: rbe_ubuntu2004
     test_flags:
       - "--test_tag_filters=-integration-test,-acceptance-test"
       - "--extra_toolchains=@buildkite_config//config:cc-toolchain"

BZLMOD_SUPPORT.md

@@ -2,10 +2,12 @@
 
 ## `rules_python` `bzlmod` support
 
-- Status: Beta
+- Status: GA
 - Full Feature Parity: No
+    - `rules_python`: Yes
+    - `rules_python_gazelle_plugin`: No (see below).
 
-Some features are missing or broken, and the public APIs are not yet stable.
+In general `bzlmod` has more features than `WORKSPACE` and users are encouraged to migrate.
 
 ## Configuration
 
@@ -27,15 +29,6 @@ A user does not use `local_path_override` stanza and would define the version in
 
 A second example, in [examples/bzlmod_build_file_generation](examples/bzlmod_build_file_generation) demonstrates the use of `bzlmod` to configure `gazelle` support for `rules_python`.
 
-## Feature parity
-
-This rule set does not have full feature partity with the older `WORKSPACE` type configuration:
-
-1. Gazelle does not support finding deps in sub-modules.  For instance we can have a dep like ` "@our_other_module//other_module/pkg:lib",` in a `py_test` definition.
-2. We have some features that are still not fully flushed out, and the user interface may change.
-
-Check ["issues"](/bazelbuild/rules_python/issues) for an up to date list.
-
 ## Differences in behavior from WORKSPACE
 
 ### Default toolchain is not the local system Python
@@ -52,10 +45,35 @@ platforms.
 If you want to use the same toolchain as what WORKSPACE used, then manually
 register the builtin Bazel Python toolchain by doing
 `register_toolchains("@bazel_tools//tools/python:autodetecting_toolchain")`.
-**IMPORTANT: this should only be done in a root module, and may intefere with
+
+Note that using this builtin Bazel toolchain is deprecated and unsupported.
+See the {obj}`runtime_env_toolchains` docs for a replacement that is marginally
+better supported.
+**IMPORTANT: this should only be done in a root module, and may interfere with
 the toolchains rules_python registers**.
 
 NOTE: Regardless of your toolchain, due to
 [#691](https://github.com/bazelbuild/rules_python/issues/691), `rules_python`
 still relies on a local Python being available to bootstrap the program before
 handing over execution to the toolchain Python.
+
+To override this behaviour see {obj}`--bootstrap_impl=script`, which switches
+to `bash`-based bootstrap on UNIX systems.
+
+### Better PyPI package downloading on bzlmod
+
+On `bzlmod` users have the option to use the `bazel_downloader` to download packages
+and work correctly when `host` platform is not the same as the `target` platform. This
+provides faster package download times and integration with the credentials helper.
+
+### Extra targets in `whl_library` repos
+
+Due to how `bzlmod` is designed and the visibility rules that it enforces, it is best to use
+the targets in the `whl` repos as they do not rely on using the `annotations` API to
+add extra targets to so-called `spoke` repos. For alternatives that should cover most of the
+existing usecases please see:
+* {bzl:obj}`py_console_script_binary` to create `entry_point` targets.
+* {bzl:obj}`whl_filegroup` to extract filegroups from the `whl` targets (e.g. `@pip//numpy:whl`)
+* {bzl:obj}`pip.override` to patch the downloaded `whl` files. Using that you
+  can change the `METADATA` of the `whl` file that will influence how
+  `rules_python` code generation behaves.

CHANGELOG.md

@@ -20,24 +20,33 @@ A brief description of the categories of changes:
 * Particular sub-systems are identified using parentheses, e.g. `(bzlmod)` or
   `(docs)`.
 
+{#v0-0-0}
 ## Unreleased
 
+[0.0.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.0.0
+
+{#v0-0-0-changed}
 ### Changed
 - Nothing yet
 
+{#v0-0-0-fixed}
 ### Fixed
 - Nothing yet
 
+{#v0-0-0-added}
 ### Added
 - Nothing yet
 
+{#v0-0-0-removed}
 ### Removed
 - Nothing yet
 
+{#v0-37-0}
 ## [0.37.0] - 2024-10-18
 
-[x.x.x]: https://github.com/bazelbuild/rules_python/releases/tag/0.37.0
+[0.37.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.37.0
 
+{#v0-37-0-changed}
 ### Changed
 * **BREAKING** `py_library` no longer puts its source files or generated pyc
   files in runfiles; it's the responsibility of consumers (e.g. binaries) to
@@ -54,6 +63,7 @@ A brief description of the categories of changes:
   {obj}`--@rules_python//python/config_settings:exec_tools_toolchain=disabled`.
 * (deps) stardoc 0.6.2 added as dependency.
 
+{#v0-37-0-fixed}
 ### Fixed
 * (bzlmod) The `python.override(minor_mapping)` now merges the default and the
   overridden versions ensuring that the resultant `minor_mapping` will always
@@ -84,6 +94,7 @@ A brief description of the categories of changes:
   outside of the `//:BUILD.bazel` file.
   Fixes [#2299](https://github.com/bazelbuild/rules_python/issues/2299).
 
+{#v0-37-0-added}
 ### Added
 * (py_wheel) Now supports `compress = (True|False)` to allow disabling
   compression to speed up development.
@@ -107,17 +118,20 @@ A brief description of the categories of changes:
 
 [20241008]: https://github.com/indygreg/python-build-standalone/releases/tag/20241008
 
+{#v0-37-0-removed}
 ### Removed
 * (precompiling) {obj}`--precompile_add_to_runfiles` has been removed.
 * (precompiling) {obj}`--pyc_collection` has been removed. The `pyc_collection`
   attribute now bases its default on {obj}`--precompile`.
 * (precompiling) The {obj}`precompile=if_generated_source` value has been removed.
 * (precompiling) The {obj}`precompile_source_retention=omit_if_generated_source` value has been removed.
 
+{#v0-36-0}
 ## [0.36.0] - 2024-09-24
 
 [0.36.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.36.0
 
+{#v0-36-0-changed}
 ### Changed
 * (gazelle): Update error messages when unable to resolve a dependency to be more human-friendly.
 * (flags) The {obj}`--python_version` flag now also returns
@@ -135,6 +149,7 @@ A brief description of the categories of changes:
   available.
 * (bazel) Minimum bazel 7 version that we test against has been bumped to `7.1`.
 
+{#v0-36-0-fixed}
 ### Fixed
 * (whl_library): Remove `--no-index` and add `--no-build-isolation` to the
   `pip install` command when installing a wheel from a local file, which happens
@@ -161,6 +176,7 @@ A brief description of the categories of changes:
 * (toolchain) The {bzl:obj}`gen_python_config_settings` has been fixed to include
   the flag_values from the platform definitions.
 
+{#v0-36-0-added}
 ### Added
 * (bzlmod): Toolchain overrides can now be done using the new
   {bzl:obj}`python.override`, {bzl:obj}`python.single_version_override` and
@@ -182,15 +198,18 @@ A brief description of the categories of changes:
 * (toolchains) Added `//python:none`, a special target for use with
   {obj}`py_exec_tools_toolchain.exec_interpreter` to treat the value as `None`.
 
+{#v0-36-0-removed}
 ### Removed
 * (toolchains): Removed accidentally exposed `http_archive` symbol from
   `python/repositories.bzl`.
 * (toolchains): An internal _is_python_config_setting_ macro has been removed.
 
+{#v0-35-0}
 ## [0.35.0] - 2024-08-15
 
 [0.35.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.35.0
 
+{#v0-35-0-changed}
 ### Changed
 * (whl_library) A better log message when the wheel is built from an sdist or
   when the wheel is downloaded using `download_only` feature to aid debugging.
@@ -204,6 +223,7 @@ A brief description of the categories of changes:
   disabling it (Requires {obj}`--bootstrap_impl=script`)
   ([#2060](https://github.com/bazelbuild/rules_python/issues/2060)).
 
+{#v0-35-0-fixed}
 ### Fixed
 * (rules) `compile_pip_requirements` now sets the `USERPROFILE` env variable on
   Windows to work around an issue where `setuptools` fails to locate the user's
@@ -245,6 +265,7 @@ A brief description of the categories of changes:
   in the same directory as the main file.
   Fixes [#1631](https://github.com/bazelbuild/rules_python/issues/1631).
 
+{#v0-35-0-added}
 ### Added
 * (rules) `compile_pip_requirements` supports multiple requirements input files as `srcs`.
 * (rules) `PYTHONSAFEPATH` is inherited from the calling environment to allow
@@ -268,10 +289,12 @@ A brief description of the categories of changes:
 [pytest_bazel]: https://pypi.org/project/pytest-bazel
 [20240726]: https://github.com/indygreg/python-build-standalone/releases/tag/20240726
 
+{#v0-34-0}
 ## [0.34.0] - 2024-07-04
 
 [0.34.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.34.0
 
+{#v0-34-0-changed}
 ### Changed
 * `protobuf`/`com_google_protobuf` dependency bumped to `v24.4`
 * (bzlmod): optimize the creation of config settings used in pip to
@@ -283,6 +306,7 @@ A brief description of the categories of changes:
   replaced by {obj}`//python/runtime_env_toolchains:all`. The old target will be
   removed in a future release.
 
+{#v0-34-0-fixed}
 ### Fixed
 * (bzlmod): When using `experimental_index_url` the `all_requirements`,
   `all_whl_requirements` and `all_data_requirements` will now only include
@@ -313,39 +337,47 @@ A brief description of the categories of changes:
 * (rules) The first element of the default outputs is now the executable again.
 * (pip) Fixed crash when pypi packages lacked a sha (e.g. yanked packages)
 
+{#v0-34-0-added}
 ### Added
 * (toolchains) {obj}`//python/runtime_env_toolchains:all`, which is a drop-in
   replacement for the "autodetecting" toolchain.
 * (gazelle) Added new `python_label_convention` and `python_label_normalization` directives. These directive
   allows altering default Gazelle label format to third-party dependencies useful for re-using Gazelle plugin
   with other rules, including `rules_pycross`. See [#1939](https://github.com/bazelbuild/rules_python/issues/1939).
 
+{#v0-34-0-removed}
 ### Removed
 * (pip): Removes the `entrypoint` macro that was replaced by `py_console_script_binary` in 0.26.0.
 
+{#v0-33-2}
 ## [0.33.2] - 2024-06-13
 
 [0.33.2]: https://github.com/bazelbuild/rules_python/releases/tag/0.33.2
 
+{#v0-33-2-fixed}
 ### Fixed
 * (toolchains) The {obj}`exec_tools_toolchain_type` is disabled by default.
   To enable it, set {obj}`--//python/config_settings:exec_tools_toolchain=enabled`.
   This toolchain must be enabled for precompilation to work. This toolchain will
   be enabled by default in a future release.
   Fixes [#1967](https://github.com/bazelbuild/rules_python/issues/1967).
 
+{#v0-33-1}
 ## [0.33.1] - 2024-06-13
 
 [0.33.1]: https://github.com/bazelbuild/rules_python/releases/tag/0.33.1
 
+{#v0-33-1-fixed}
 ### Fixed
 * (py_binary) Fix building of zip file when using `--build_python_zip`
   argument. Fixes [#1954](https://github.com/bazelbuild/rules_python/issues/1954).
 
+{#v0-33-0}
 ## [0.33.0] - 2024-06-12
 
 [0.33.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.33.0
 
+{#v0-33-0-changed}
 ### Changed
 * (deps) Upgrade the `pip_install` dependencies to pick up a new version of pip.
 * (toolchains) Optional toolchain dependency: `py_binary`, `py_test`, and
@@ -374,6 +406,7 @@ A brief description of the categories of changes:
   `python_{version}_host` keys if you would like to have access to a Python
   interpreter that can be used in a repository rule context.
 
+{#v0-33-0-fixed}
 ### Fixed
 * (gazelle) Remove `visibility` from `NonEmptyAttr`.
   Now empty(have no `deps/main/srcs/imports` attr) `py_library/test/binary` rules will
@@ -406,6 +439,7 @@ A brief description of the categories of changes:
 * (doc) Fix the `WORKSPACE` requirement vendoring example. Fixes
   [#1918](https://github.com/bazelbuild/rules_python/issues/1918).
 
+{#v0-33-0-added}
 ### Added
 * (rules) Precompiling Python source at build time is available. but is
   disabled by default, for now. Set
@@ -459,21 +493,25 @@ A brief description of the categories of changes:
 
 [precompile-docs]: /precompiling
 
+{#v0-32-2}
 ## [0.32.2] - 2024-05-14
 
 [0.32.2]: https://github.com/bazelbuild/rules_python/releases/tag/0.32.2
 
+{#v0-32-2-fixed}
 ### Fixed
 
 * Workaround existence of infinite symlink loops on case insensitive filesystems when targeting linux platforms with recent Python toolchains. Works around an upstream [issue][indygreg-231]. Fixes [#1800][rules_python_1800].
 
 [indygreg-231]: https://github.com/indygreg/python-build-standalone/issues/231
 [rules_python_1800]: https://github.com/bazelbuild/rules_python/issues/1800
 
+{#v0-32-0}
 ## [0.32.0] - 2024-05-12
 
 [0.32.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.32.0
 
+{#v0-32-0-changed}
 ### Changed
 
 * (bzlmod): The `MODULE.bazel.lock` `whl_library` rule attributes are now

CONTRIBUTING.md

@@ -194,8 +194,13 @@ The general process is:
    of. The API for the control mechanism can be removed in this release.
 
 Note that the `+1` and `+2` releases are just examples; the steps are not
-required to happen in immedially subsequent releases.
+required to happen in immediately subsequent releases.
 
+Once The first major version is released, the process will be:
+1. In `N.M.0` we introduce the new behaviour, but it is disabled by a feature flag.
+2. In `N.M+1.0` we may choose the behaviour to become the default if it is not too
+   disruptive.
+3. In `N+1.0.0` we get rid of the old behaviour.
 
 ### How to control breaking changes