Merge branch 'main' of https://github.com/bazelbuild/rules_python into feat.interpreter.args

Author: rickeylev

CHANGELOG.md

@@ -91,6 +91,9 @@ Unreleased changes template.
 * (pypi) Direct HTTP urls for wheels and sdists are now supported when using
   {obj}`experimental_index_url` (bazel downloader).
   Partially fixes [#2363](https://github.com/bazelbuild/rules_python/issues/2363).
+* (rules) APIs for creating custom rules based on the core py_binary, py_test,
+  and py_library rules
+  ([#1647](https://github.com/bazelbuild/rules_python/issues/1647))
 * (rules) Added {obj}`interpreter_args` attribute to `py_binary` and `py_test`,
   which allows pass arguments to the interpreter before the regular args.
 

docs/BUILD.bazel

@@ -100,6 +100,8 @@ sphinx_stardocs(
         "//python:py_test_bzl",
         "//python:repositories_bzl",
         "//python/api:api_bzl",
+        "//python/api:executables_bzl",
+        "//python/api:libraries_bzl",
         "//python/cc:py_cc_toolchain_bzl",
         "//python/cc:py_cc_toolchain_info_bzl",
         "//python/entry_points:py_console_script_binary_bzl",

docs/_includes/volatile_api.md

@@ -0,0 +1,5 @@
+:::{important}
+
+**Public, but volatile, API.** Some parts are stable, while others are
+implementation details and may change more frequently.
+:::

docs/extending.md

@@ -0,0 +1,143 @@
+# Extending the rules
+
+:::{important}
+**This is public, but volatile, functionality.**
+
+Extending and customizing the rules is supported functionality, but with weaker
+backwards compatibility guarantees, and is not fully subject to the normal
+backwards compatibility procedures and policies. It's simply not feasible to
+support every possible customization with strong backwards compatibility
+guarantees.
+:::
+
+Because of the rich ecosystem of tools and variety of use cases, APIs are
+provided to make it easy to create custom rules using the existing rules as a
+basis. This allows implementing behaviors that aren't possible using
+wrapper macros around the core rules, and can make certain types of changes
+much easier and transparent to implement.
+
+:::{note}
+It is not required to extend a core rule. The minimum requirement for a custom
+rule is to return the appropriate provider (e.g. {bzl:obj}`PyInfo` etc).
+Extending the core rules is most useful when you want all or most of the
+behavior of a core rule.
+:::
+
+Follow or comment on https://github.com/bazelbuild/rules_python/issues/1647
+for the development of APIs to support custom derived rules.
+
+## Creating custom rules
+
+Custom rules can be created using the core rules as a basis by using their rule
+builder APIs.
+
+* [`//python/apis:executables.bzl`](#python-apis-executables-bzl): builders for
+  executables.
+* [`//python/apis:libraries.bzl`](#python-apis-libraries-bzl): builders for
+  libraries.
+
+These builders create {bzl:obj}`ruleb.Rule` objects, which are thin
+wrappers around the keyword arguments eventually passed to the `rule()`
+function. These builder APIs give access to the _entire_ rule definition and
+allow arbitrary modifications.
+
+This is level of control is powerful, but also volatile. A rule definition
+contains many details that _must_ change as the implementation changes. What
+is more or less likely to change isn't known in advance, but some general
+rules are:
+
+* Additive behavior to public attributes will be less prone to breaking.
+* Internal attributes that directly support a public attribute are likely
+  reliable.
+* Internal attributes that support an action are more likely to change.
+* Rule toolchains are moderately stable (toolchains are mostly internal to
+  how a rule works, but custom toolchains are supported).
+
+## Example: validating a source file
+
+In this example, we derive from `py_library` a custom rule that verifies source
+code contains the word "snakes". It does this by:
+
+* Adding an implicit dependency on a checker program
+* Calling the base implementation function
+* Running the checker on the srcs files
+* Adding the result to the `_validation` output group (a special output
+  group for validation behaviors).
+
+To users, they can use `has_snakes_library` the same as `py_library`. The same
+is true for other targets that might consume the rule.
+
+```
+load("@rules_python//python/api:libraries.bzl", "libraries")
+load("@rules_python//python/api:attr_builders.bzl", "attrb")
+
+def _has_snakes_impl(ctx, base):
+    providers = base(ctx)
+
+    out = ctx.actions.declare_file(ctx.label.name + "_snakes.check")
+    ctx.actions.run(
+        inputs = ctx.files.srcs,
+        outputs = [out],
+        executable = ctx.attr._checker[DefaultInfo].files_to_run,
+        args = [out.path] + [f.path for f in ctx.files.srcs],
+    )
+    prior_ogi = None
+    for i, p in enumerate(providers):
+        if type(p) == "OutputGroupInfo":
+            prior_ogi = (i, p)
+            break
+    if prior_ogi:
+        groups = {k: getattr(prior_ogi[1], k) for k in dir(prior_ogi)}
+        if "_validation" in groups:
+            groups["_validation"] = depset([out], transitive=groups["_validation"])
+        else:
+            groups["_validation"] = depset([out])
+        providers[prior_ogi[0]] = OutputGroupInfo(**groups)
+    else:
+        providers.append(OutputGroupInfo(_validation=depset([out])))
+    return providers
+
+def create_has_snakes_rule():
+    r = libraries.py_library_builder()
+    base_impl = r.implementation()
+    r.set_implementation(lambda ctx: _has_snakes_impl(ctx, base_impl))
+    r.attrs["_checker"] = attrb.Label(
+        default="//:checker",
+        executable = True,
+    )
+    return r.build()
+has_snakes_library = create_has_snakes_rule()
+```
+
+## Example: adding transitions
+
+In this example, we derive from `py_binary` to force building for a particular
+platform. We do this by:
+
+* Adding an additional output to the rule's cfg
+* Calling the base transition function
+* Returning the new transition outputs
+
+```starlark
+
+load("@rules_python//python/api:executables.bzl", "executables")
+
+def _force_linux_impl(settings, attr, base_impl):
+    settings = base_impl(settings, attr)
+    settings["//command_line_option:platforms"] = ["//my/platforms:linux"]
+    return settings
+
+def create_rule():
+    r = executables.py_binary_rule_builder()
+    base_impl = r.cfg.implementation()
+    r.cfg.set_implementation(
+        lambda settings, attr: _force_linux_impl(settings, attr, base_impl)
+    )
+    r.cfg.add_output("//command_line_option:platforms")
+    return r.build()
+
+py_linux_binary = create_linux_binary_rule()
+```
+
+Users can then use `py_linux_binary` the same as a regular py_binary. It will
+act as if `--platforms=//my/platforms:linux` was specified when building it.

docs/index.md

@@ -101,6 +101,7 @@ pip
 coverage
 precompiling
 gazelle
+Extending <extending>
 Contributing <contributing>
 support
 Changelog <changelog>

python/api/BUILD.bazel

@@ -25,6 +25,26 @@ bzl_library(
     deps = ["//python/private/api:api_bzl"],
 )
 
+bzl_library(
+    name = "executables_bzl",
+    srcs = ["executables.bzl"],
+    visibility = ["//visibility:public"],
+    deps = [
+        "//python/private:py_binary_rule_bzl",
+        "//python/private:py_executable_bzl",
+        "//python/private:py_test_rule_bzl",
+    ],
+)
+
+bzl_library(
+    name = "libraries_bzl",
+    srcs = ["libraries.bzl"],
+    visibility = ["//visibility:public"],
+    deps = [
+        "//python/private:py_library_bzl",
+    ],
+)
+
 filegroup(
     name = "distribution",
     srcs = glob(["**"]),

python/api/executables.bzl

@@ -0,0 +1,31 @@
+# Copyright 2025 The Bazel Authors. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+{#python-apis-executables-bzl}
+Loading-phase APIs specific to executables (binaries/tests).
+
+:::{versionadded} VERSION_NEXT_FEATURE
+:::
+"""
+
+load("//python/private:py_binary_rule.bzl", "create_py_binary_rule_builder")
+load("//python/private:py_executable.bzl", "create_executable_rule_builder")
+load("//python/private:py_test_rule.bzl", "create_py_test_rule_builder")
+
+executables = struct(
+    py_binary_rule_builder = create_py_binary_rule_builder,
+    py_test_rule_builder = create_py_test_rule_builder,
+    executable_rule_builder = create_executable_rule_builder,
+)

python/api/libraries.bzl

@@ -0,0 +1,27 @@
+# Copyright 2025 The Bazel Authors. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+{#python-apis-libraries-bzl}
+Loading-phase APIs specific to libraries.
+
+:::{versionadded} VERSION_NEXT_FEATURE
+:::
+"""
+
+load("//python/private:py_library.bzl", "create_py_library_rule_builder")
+
+libraries = struct(
+    py_library_rule_builder = create_py_library_rule_builder,
+)

python/private/BUILD.bazel

@@ -427,6 +427,7 @@ bzl_library(
         ":attributes_bzl",
         ":common_bzl",
         ":flags_bzl",
+        ":precompile_bzl",
         ":py_cc_link_params_info_bzl",
         ":py_internal_bzl",
         ":rule_builders_bzl",
@@ -446,8 +447,6 @@ bzl_library(
     name = "py_library_rule_bzl",
     srcs = ["py_library_rule.bzl"],
     deps = [
-        ":common_bzl",
-        ":precompile_bzl",
         ":py_library_bzl",
     ],
 )

python/private/attr_builders.bzl

@@ -12,7 +12,11 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-"""Builders for creating attributes et al."""
+"""Builders for creating attributes et al.
+
+:::{versionadded} VERSION_NEXT_FEATURE
+:::
+"""
 
 load("@bazel_skylib//lib:types.bzl", "types")
 load(