feat: allow custom platform when overriding (#2880)

This basically allows using any python-build-standalone archive and
using it
if custom flags are set. This is done through the
`single_version_platform_override()`
API, because such archives are inherently version and platform specific.

Key changes:
* The `platform` arg can be any value (mostly; it ends up in repo names)
* Added `target_compatible_with` and `target_settings` args, which
become the
  settings used on the generated toolchain() definition.

The platform settings are version specific, i.e. the key
`(python_version, platform)`
is what maps to the TCW/TS values.

If an existing platform is used, it'll override the defaults that
normally come
from the PLATFORMS global for the particular version. If a new platform
is used,
it creates a new platform entry with those settings.

Along the way:
* Added various docs about internal variables so they're easier to grok
at a glance.

Work towards #2081

Author: rickeylev

CHANGELOG.md

@@ -107,6 +107,10 @@ END_UNRELEASED_TEMPLATE
   Set the `RULES_PYTHON_ENABLE_PIPSTAR=1` environment variable to enable it.
 * (utils) Add a way to run a REPL for any `rules_python` target that returns
   a `PyInfo` provider.
+* (toolchains) Arbitrary python-build-standalone runtimes can be registered
+  and activated with custom flags. See the [Registering custom runtimes]
+  docs and {obj}`single_version_platform_override()` API docs for more
+  information.
 
 {#v0-0-0-removed}
 ### Removed

MODULE.bazel

@@ -125,6 +125,22 @@ dev_python.override(
     register_all_versions = True,
 )
 
+# For testing an arbitrary runtime triggered by a custom flag.
+# See //tests/toolchains:custom_platform_toolchain_test
+dev_python.single_version_platform_override(
+    platform = "linux-x86-install-only-stripped",
+    python_version = "3.13.1",
+    sha256 = "56817aa976e4886bec1677699c136cb01c1cdfe0495104c0d8ef546541864bbb",
+    target_compatible_with = [
+        "@platforms//os:linux",
+        "@platforms//cpu:x86_64",
+    ],
+    target_settings = [
+        "@@//tests/support:is_custom_runtime_linux-x86-install-only-stripped",
+    ],
+    urls = ["https://github.com/astral-sh/python-build-standalone/releases/download/20250115/cpython-3.13.1+20250115-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz"],
+)
+
 dev_pip = use_extension(
     "//python/extensions:pip.bzl",
     "pip",

docs/toolchains.md

@@ -243,6 +243,73 @@ existing attributes:
 * Adding additional Python versions via {bzl:obj}`python.single_version_override` or
   {bzl:obj}`python.single_version_platform_override`.
 
+### Registering custom runtimes
+
+Because the python-build-standalone project has _thousands_ of prebuilt runtimes
+available, rules_python only includes popular runtimes in its built in
+configurations. If you want to use a runtime that isn't already known to
+rules_python then {obj}`single_version_platform_override()` can be used to do
+so. In short, it allows specifying an arbitrary URL and using custom flags
+to control when a runtime is used.
+
+In the example below, we register a particular python-build-standalone runtime
+that is activated for Linux x86 builds when the custom flag
+`--//:runtime=my-custom-runtime` is set.
+
+```
+# File: MODULE.bazel
+bazel_dep(name = "bazel_skylib", version = "1.7.1.")
+bazel_dep(name = "rules_python", version = "1.5.0")
+python = use_extension("@rules_python//python/extensions:python.bzl", "python")
+python.single_version_platform_override(
+    platform = "my-platform",
+    python_version = "3.13.3",
+    sha256 = "01d08b9bc8a96698b9d64c2fc26da4ecc4fa9e708ce0a34fb88f11ab7e552cbd",
+    os_name = "linux",
+    arch = "x86_64",
+    target_settings = [
+        "@@//:runtime=my-custom-runtime",
+    ],
+    urls = ["https://github.com/astral-sh/python-build-standalone/releases/download/20250409/cpython-3.13.3+20250409-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz"],
+)
+# File: //:BUILD.bazel
+load("@bazel_skylib//rules:common_settings.bzl", "string_flag")
+string_flag(
+    name = "custom_runtime",
+    build_setting_default = "",
+)
+config_setting(
+    name = "is_custom_runtime_linux-x86-install-only-stripped",
+    flag_values = {
+        ":custom_runtime": "linux-x86-install-only-stripped",
+    },
+)
+```
+
+Notes:
+- While any URL and archive can be used, it's assumed their content looks how
+  a python-build-standalone archive looks.
+- A "version aware" toolchain is registered, which means the Python version flag
+  must also match (e.g. `--@rules_python//python/config_settings:python_version=3.13.3`
+  must be set -- see `minor_mapping` and `is_default` for controls and docs
+  about version matching and selection).
+- The `target_compatible_with` attribute can be used to entirely specify the
+  arg of the same name the toolchain uses.
+- The labels in `target_settings` must be absolute; `@@` refers to the main repo.
+- The `target_settings` are `config_setting` targets, which means you can
+  customize how matching occurs.
+
+:::{seealso}
+See {obj}`//python/config_settings` for flags rules_python already defines
+that can be used with `target_settings`. Some particular ones of note are:
+{flag}`--py_linux_libc` and {flag}`--py_freethreaded`, among others.
+:::
+
+:::{versionadded} VERSION_NEXT_FEATURE
+Added support for custom platform names, `target_compatible_with`, and
+`target_settings` with `single_version_platform_override`.
+:::
+
 ### Using defined toolchains from WORKSPACE
 
 It is possible to use toolchains defined in `MODULE.bazel` in `WORKSPACE`. For example

internal_dev_setup.bzl

@@ -42,7 +42,7 @@ def rules_python_internal_setup():
         toolchain_platform_keys = {},
         toolchain_python_versions = {},
         toolchain_set_python_version_constraints = {},
-        base_toolchain_repo_names = [],
+        host_compatible_repo_names = [],
     )
 
     runtime_env_repo(name = "rules_python_runtime_env_tc_info")

python/BUILD.bazel

@@ -247,6 +247,7 @@ bzl_library(
     name = "versions_bzl",
     srcs = ["versions.bzl"],
     visibility = ["//:__subpackages__"],
+    deps = ["//python/private:platform_info_bzl"],
 )
 
 # NOTE: Remember to add bzl_library targets to //tests:bzl_libraries

python/private/BUILD.bazel

@@ -241,11 +241,17 @@ bzl_library(
     ],
 )
 
+bzl_library(
+    name = "platform_info_bzl",
+    srcs = ["platform_info.bzl"],
+)
+
 bzl_library(
     name = "python_bzl",
     srcs = ["python.bzl"],
     deps = [
         ":full_version_bzl",
+        ":platform_info_bzl",
         ":python_register_toolchains_bzl",
         ":pythons_hub_bzl",
         ":repo_utils_bzl",

python/private/platform_info.bzl

@@ -0,0 +1,34 @@
+"""Helper to define a struct used to define platform metadata."""
+
+def platform_info(
+        *,
+        compatible_with = [],
+        flag_values = {},
+        target_settings = [],
+        os_name,
+        arch):
+    """Creates a struct of platform metadata.
+
+    This is just a helper to ensure structs are created the same and
+    the meaning/values are documented.
+
+    Args:
+        compatible_with: list[str], where the values are string labels. These
+            are the target_compatible_with values to use with the toolchain
+        flag_values: dict[str|Label, Any] of config_setting.flag_values
+            compatible values. DEPRECATED -- use target_settings instead
+        target_settings: list[str], where the values are string labels. These
+            are the target_settings values to use with the toolchain.
+        os_name: str, the os name; must match the name used in `@platfroms//os`
+        arch: str, the cpu name; must match the name used in `@platforms//cpu`
+
+    Returns:
+        A struct with attributes and values matching the args.
+    """
+    return struct(
+        compatible_with = compatible_with,
+        flag_values = flag_values,
+        target_settings = target_settings,
+        os_name = os_name,
+        arch = arch,
+    )

python/private/py_repositories.bzl

@@ -47,7 +47,7 @@ def py_repositories():
         toolchain_platform_keys = {},
         toolchain_python_versions = {},
         toolchain_set_python_version_constraints = {},
-        base_toolchain_repo_names = [],
+        host_compatible_repo_names = [],
     )
     http_archive(
         name = "bazel_skylib",