Introduce runtime_toolchain_type (#3859)

According to discussions in #3854 having two toolchains of the same type
for different things is troublesome. It's better to have separate
runtime as well as compile toolchains.

This commit creates a new runtime_toolchain_type and registers
toolchains without execution constraints for this type. Once merged,
rules_ts can start consuming the new toolchain type in its js_binary
rule to ensure the correct Node for the correct target environment is
selected.

Fixed [Bug]: Execution toolchain defined without
`target_compatible_with` makes it a candidate to selection

Fixes #3854
Work towards #3795

## PR Checklist

Please check if your PR fulfills the following requirements:

- [x] Tests for the changes have been added (for bug fixes / features)
- [x] Docs have been added / updated (for bug fixes / features)


## PR Type

What kind of change does this PR introduce?

<!-- Please check the one that applies to this PR using "x". -->

- [x] Bugfix
- [x] Feature (please, look at the "Scope of the project" section in the
README.md file)
- [x] Documentation content changes


## Does this PR introduce a breaking change?

- [ ] Yes
- [x] No

It tries to remain compatible and support existing consumers.

## Other information

This is an alternative to #3800.

Author: guw

MODULE.bazel

@@ -10,6 +10,9 @@ bazel_dep(name = "bazel_lib", version = "3.0.0-beta.1")
 bazel_dep(name = "bazel_skylib", version = "1.4.1")
 bazel_dep(name = "platforms", version = "0.0.5")
 
+# workaround for https://github.com/bazelbuild/bazel/issues/25124
+bazel_dep(name = "zlib", version = "1.3.1.bcr.6", dev_dependency = True)
+
 node = use_extension("@rules_nodejs//nodejs:extensions.bzl", "node")
 
 # Note, this gets the default version of Node.js from
@@ -19,37 +22,37 @@ use_repo(node, "nodejs_toolchains")
 
 # Toolchain registration under bzlmod should match the order of WORKSPACE registration
 # which is the order specified in the PLATFORMS dict https://github.com/bazel-contrib/rules_nodejs/blob/4c373209b058d46f2a5f9ab9f8abf11b161ae459/nodejs/private/nodejs_toolchains_repo.bzl#L20.
-# For each platform, `:<PLATFORM>_toolchain_target` should be registered before `:<PLATFORM>_toolchain`,
+# For each platform, `:<PLATFORM>_runtime_toolchain` should be registered before `:<PLATFORM>_toolchain`,
 # https://github.com/bazel-contrib/rules_nodejs/blob/4c373209b058d46f2a5f9ab9f8abf11b161ae459/nodejs/repositories.bzl#L461/.
 # See https://github.com/bazelbuild/bazel/issues/19645 and https://github.com/bazel-contrib/rules_nodejs/pull/3750 for more context.
-register_toolchains("@nodejs_toolchains//:linux_amd64_toolchain_target")
+register_toolchains("@nodejs_toolchains//:linux_amd64_runtime_toolchain")
 
 register_toolchains("@nodejs_toolchains//:linux_amd64_toolchain")
 
-register_toolchains("@nodejs_toolchains//:linux_arm64_toolchain_target")
+register_toolchains("@nodejs_toolchains//:linux_arm64_runtime_toolchain")
 
 register_toolchains("@nodejs_toolchains//:linux_arm64_toolchain")
 
-register_toolchains("@nodejs_toolchains//:linux_s390x_toolchain_target")
+register_toolchains("@nodejs_toolchains//:linux_s390x_runtime_toolchain")
 
 register_toolchains("@nodejs_toolchains//:linux_s390x_toolchain")
 
-register_toolchains("@nodejs_toolchains//:linux_ppc64le_toolchain_target")
+register_toolchains("@nodejs_toolchains//:linux_ppc64le_runtime_toolchain")
 
 register_toolchains("@nodejs_toolchains//:linux_ppc64le_toolchain")
 
-register_toolchains("@nodejs_toolchains//:darwin_amd64_toolchain_target")
+register_toolchains("@nodejs_toolchains//:darwin_amd64_runtime_toolchain")
 
 register_toolchains("@nodejs_toolchains//:darwin_amd64_toolchain")
 
-register_toolchains("@nodejs_toolchains//:darwin_arm64_toolchain_target")
+register_toolchains("@nodejs_toolchains//:darwin_arm64_runtime_toolchain")
 
 register_toolchains("@nodejs_toolchains//:darwin_arm64_toolchain")
 
-register_toolchains("@nodejs_toolchains//:windows_amd64_toolchain_target")
+register_toolchains("@nodejs_toolchains//:windows_amd64_runtime_toolchain")
 
 register_toolchains("@nodejs_toolchains//:windows_amd64_toolchain")
 
-register_toolchains("@nodejs_toolchains//:windows_arm64_toolchain_target")
+register_toolchains("@nodejs_toolchains//:windows_arm64_runtime_toolchain")
 
 register_toolchains("@nodejs_toolchains//:windows_arm64_toolchain")

docs/Toolchains.md

@@ -3,6 +3,16 @@
 API docs for [Toolchain](https://docs.bazel.build/versions/main/toolchains.html) support.
 
 When you call `nodejs_register_toolchains()` in your `WORKSPACE` file it will setup a node toolchain for executing tools on all currently supported platforms.
+In `bzlmod` default toolchains are registered automatically when you depend on `rules_nodejs`.
+
+There are two toolchain types:
+
+1) The transpilation toolchain, which provides the Node runtime used to execute the transpiler (and type checker), as well as various helper tools and settings.
+   (`@rules_nodejs//nodejs:toolchain_type`)
+2) The Node runtime that executable Node outputs (e.g., js_binary) will run on.
+   (`@rules_nodejs//nodejs:runtime_toolchain_type`)
+
+See [//nodjes/BUILD.bazel](https://github.com/bazel-contrib/rules_nodejs/blob/main/nodejs/BUILD.bazel) for details.
 
 If you have an advanced use-case and want to use a version of node not supported by this repository, you can also register your own toolchains.
 
@@ -13,9 +23,9 @@ To run a custom toolchain (i.e., to run a node binary not supported by the built
 1) A rule which can build or load a node binary from your repository
    (a checked-in binary or a build using a relevant [`rules_foreign_cc` build rule](https://bazelbuild.github.io/rules_foreign_cc/) will do nicely).
 2) A [`nodejs_toolchain` rule](Core.html#nodejs_toolchain) which depends on your binary defined in step 1 as its `node`.
-3) A [`toolchain` rule](https://bazel.build/reference/be/platform#toolchain) that depends on your `nodejs_toolchain` rule defined in step 2 as its `toolchain`
-   and on `@rules_nodejs//nodejs:toolchain_type` as its `toolchain_type`. Make sure to define appropriate platform restrictions as described in the
-   documentation for the `toolchain` rule.
+3) Two [`toolchain` rules](https://bazel.build/reference/be/platform#toolchain) each depends on your `nodejs_toolchain` rule defined in step 2 as its `toolchain`
+   and one on `@rules_nodejs//nodejs:toolchain_type` as its `toolchain_type` and the other one  `@rules_nodejs//nodejs:runtime_toolchain_type`.
+   Make sure to define appropriate platform restrictions as described in the documentation for the `toolchain` rule.
 4) A call to [the `register_toolchains` function](https://bazel.build/rules/lib/globals#register_toolchains) in your `WORKSPACE`
    that refers to the `toolchain` rule defined in step 3.
 

e2e/smoke/BUILD.bazel

@@ -9,6 +9,12 @@ not_windows = select({
     "//conditions:default": [],
 })
 
+not_silicon = select({
+    # There isn't a published arm64 binary for Node 15
+    "@platforms//cpu:arm64": ["@platforms//:incompatible"],
+    "//conditions:default": [],
+})
+
 # Trivial test fixture: a nodejs program that writes to a file
 write_file(
     name = "js",
@@ -67,7 +73,6 @@ genrule(
     outs = ["actual1"],
     cmd = "$(NODE_PATH) $(execpath some.js) $@",
     toolchains = ["@nodejs_toolchains//:resolved_toolchain"],
-    tools = ["@nodejs_toolchains//:resolved_toolchain"],
 )
 
 diff_test(
@@ -253,7 +258,9 @@ my_nodejs(
     # you can also just provide an individual toolchain if you don't want to download them all
     toolchain = select({
         "@bazel_tools//src/conditions:linux_x86_64": "@node17_linux_amd64//:toolchain",
-        "@bazel_tools//src/conditions:darwin": "@node17_darwin_amd64//:toolchain",
+        "@bazel_tools//src/conditions:linux_aarch64": "@node17_linux_arm64//:toolchain",
+        "@bazel_tools//src/conditions:darwin_x86_64": "@node17_darwin_amd64//:toolchain",
+        "@bazel_tools//src/conditions:darwin_arm64": "@node17_darwin_arm64//:toolchain",
         "@bazel_tools//src/conditions:windows": "@node17_windows_amd64//:toolchain",
     }),
 )
@@ -273,7 +280,9 @@ my_nodejs(
     # you can also just provide an individual toolchain if you don't want to download them all
     toolchain = select({
         "@bazel_tools//src/conditions:linux_x86_64": "@nodejs_linux_amd64//:toolchain",
-        "@bazel_tools//src/conditions:darwin": "@nodejs_darwin_amd64//:toolchain",
+        "@bazel_tools//src/conditions:linux_aarch64": "@nodejs_linux_arm64//:toolchain",
+        "@bazel_tools//src/conditions:darwin_x86_64": "@nodejs_darwin_amd64//:toolchain",
+        "@bazel_tools//src/conditions:darwin_arm64": "@nodejs_darwin_arm64//:toolchain",
         "@bazel_tools//src/conditions:windows": "@nodejs_windows_amd64//:toolchain",
     }),
 )
@@ -288,11 +297,12 @@ my_nodejs(
     name = "run_15",
     out = "thing_toolchain_15",
     entry_point = "version.js",
+    target_compatible_with = not_silicon,
     # using the select statement will download toolchains for all three platforms
     # you can also just provide an individual toolchain if you don't want to download them all
     toolchain = select({
         "@bazel_tools//src/conditions:linux_x86_64": "@node15_linux_amd64//:toolchain",
-        "@bazel_tools//src/conditions:darwin": "@node15_darwin_amd64//:toolchain",
+        "@bazel_tools//src/conditions:darwin_x86_64": "@node15_darwin_amd64//:toolchain",
         "@bazel_tools//src/conditions:windows": "@node15_windows_amd64//:toolchain",
     }),
 )

e2e/smoke/MODULE.bazel

@@ -30,10 +30,14 @@ use_repo(
     "node15_linux_amd64",
     "node15_windows_amd64",
     "node17_darwin_amd64",
+    "node17_darwin_arm64",
     "node17_linux_amd64",
+    "node17_linux_arm64",
     "node17_windows_amd64",
     "nodejs_darwin_amd64",
+    "nodejs_darwin_arm64",
     "nodejs_linux_amd64",
+    "nodejs_linux_arm64",
     "nodejs_toolchains",
     "nodejs_windows_amd64",
 )

nodejs/BUILD.bazel

@@ -2,6 +2,7 @@
 load("@bazel_lib//:bzl_library.bzl", "bzl_library")
 load("//nodejs/private:nodejs_toolchains_repo.bzl", "PLATFORMS")
 load("//nodejs/private:user_build_settings.bzl", "user_args")
+load(":node_toolchain_alias.bzl", "node_runtime_alias", "node_toolchain_alias")
 
 package(default_visibility = ["//visibility:public"])
 
@@ -25,23 +26,43 @@ bzl_library(
     ],
 )
 
-bzl_library(
-    name = "toolchain",
-    srcs = ["toolchain.bzl"],
-)
-
-bzl_library(
-    name = "extensions",
-    srcs = ["extensions.bzl"],
-    deps = ["repositories"],
-)
-
 # This is the target rule authors should put in their "toolchains"
 # attribute in order to get a node interpreter for the correct
 # platform.
 # See https://docs.bazel.build/versions/main/toolchains.html#writing-rules-that-use-toolchains
+# A single binary distribution of a Node provides two different types of toolchains from the
+# perspective of Bazel:
+
+# (1) The transpilation toolchain, which provides the Node runtime used to execute the transpiler
+# (and type checker), as well as various helper tools and settings.
+#
+# Toolchains of this type typically have constraints on the execution platform so that their Node
+# runtime can run the transpiler, but not on the target platform as Node transpilation outputs are
+# platform independent.
+#
+# Obtain the associated NodeInfo via:
+#   ctx.toolchains["@bazel_tools//tools/jdk:toolchain_type"].nodeinfo
 toolchain_type(name = "toolchain_type")
 
+# (2) The Node runtime that executable Node outputs (e.g., js_binary) will run on.
+#
+# Toolchains of this type typically have constraints on the target platform so that the runtime's
+# native 'node' binary can be run there, but not on the execution platform as building an executable
+# Node target only requires copying or symlinking the runtime, which can be done on any platform.
+#
+# Obtain the associated NodeRuntimeInfo via:
+#   ctx.toolchains["@bazel_tools//tools/jdk:runtime_toolchain_type"].nodeinfo
+toolchain_type(name = "runtime_toolchain_type")
+
+# Points to toolchain[":runtime_toolchain_type"]
+# Use this for executing and packaging Node applications for target platform (eg., js_binary, js_test or js_image_oci).
+node_runtime_alias(name = "current_node_runtime")
+
+# Points to toolchain[":toolchain_type"]
+# Use this for tools (eg., when action execution is needed).
+node_toolchain_alias(name = "current_node_toolchain")
+
+# The platforms that are supported by the Node toolchains.
 [
     platform(
         name = key,

nodejs/node_toolchain_alias.bzl

@@ -0,0 +1,53 @@
+# Copyright 2019 The Bazel Authors. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Node toolchain aliases using toolchain resolution."""
+
+load(":semantics.bzl", "semantics")
+
+def _node_runtime_alias(ctx):
+    """Implementation of node_runtime_alias using toolchain resolution."""
+    toolchain_info = ctx.toolchains[semantics.NODE_RUNTIME_TOOLCHAIN_TYPE]
+    toolchain = toolchain_info.nodeinfo
+    template_variable_info = toolchain_info.template_variables
+    default_info = toolchain_info.default
+    return [
+        toolchain_info,
+        toolchain,
+        template_variable_info,
+        default_info,
+    ]
+
+node_runtime_alias = rule(
+    implementation = _node_runtime_alias,
+    toolchains = [semantics.NODE_RUNTIME_TOOLCHAIN],
+)
+
+def _node_toolchain_alias(ctx):
+    """An implementation of node_toolchain_alias using toolchain resolution."""
+    toolchain_info = ctx.toolchains[semantics.NODE_TOOLCHAIN_TYPE]
+    toolchain = toolchain_info.nodeinfo
+    template_variable_info = toolchain_info.template_variables
+    default_info = toolchain_info.default
+    return [
+        toolchain_info,
+        toolchain,
+        template_variable_info,
+        default_info,
+    ]
+
+node_toolchain_alias = rule(
+    implementation = _node_toolchain_alias,
+    toolchains = [semantics.NODE_TOOLCHAIN],
+)

nodejs/private/nodejs_toolchains_repo.bzl

@@ -69,55 +69,42 @@ PLATFORMS = {
 }
 
 def _nodejs_toolchains_repo_impl(repository_ctx):
-    # Expose a concrete toolchain which is the result of Bazel resolving the toolchain
-    # for the execution or target platform.
-    # Workaround for https://github.com/bazelbuild/bazel/issues/14009
-    starlark_content = """# Generated by nodejs_toolchains_repo.bzl
-
-# Forward all the providers
-def _resolved_toolchain_impl(ctx):
-    toolchain_info = ctx.toolchains["@rules_nodejs//nodejs:toolchain_type"]
-    return [
-        toolchain_info,
-        toolchain_info.default,
-        toolchain_info.nodeinfo,
-        toolchain_info.template_variables,
-    ]
-
-# Copied from java_toolchain_alias
-# https://cs.opensource.google/bazel/bazel/+/master:tools/jdk/java_toolchain_alias.bzl
-resolved_toolchain = rule(
-    implementation = _resolved_toolchain_impl,
-    toolchains = ["@rules_nodejs//nodejs:toolchain_type"],
-)
-"""
-    repository_ctx.file("defs.bzl", starlark_content)
-
-    build_content = """# Generated by nodejs_toolchains_repo.bzl
+    # TODO(7.0) Drop support for deprecated alias
+    build_content = '''# Generated by nodejs_toolchains_repo.bzl
 #
 # These can be registered in the workspace file or passed to --extra_toolchains flag.
 # By default all these toolchains are registered by the nodejs_register_toolchains macro
 # so you don't normally need to interact with these targets.
 
-load(":defs.bzl", "resolved_toolchain")
-
-resolved_toolchain(name = "resolved_toolchain", visibility = ["//visibility:public"])
+alias(
+    name = "resolved_toolchain",
+    actual = "@rules_nodejs//nodejs:current_node_runtime",
+    deprecation = """
+Use one of the following instead:
+- @rules_nodejs//nodejs:current_node_runtime
+- @rules_nodejs//nodejs:current_host_node_runtime
+- @rules_nodejs//nodejs:current_node_toolchain
+See https://github.com/bazel-contrib/rules_nodejs/issues/3795.
+""",
+    visibility = ["//visibility:public"],
+)
 
-"""
+'''
 
     for [platform, meta] in PLATFORMS.items():
         build_content += """
 toolchain(
     name = "{platform}_toolchain",
     exec_compatible_with = {compatible_with},
+    target_compatible_with = {compatible_with}, # prevent Node from this toolchain being bundled by js_image_oci to incompatible target platforms (https://github.com/bazel-contrib/rules_nodejs/issues/3854)
     toolchain = "@{user_node_repository_name}_{platform}//:toolchain",
     toolchain_type = "@rules_nodejs//nodejs:toolchain_type",
 )
 toolchain(
-    name = "{platform}_toolchain_target",
+    name = "{platform}_runtime_toolchain",
     target_compatible_with = {compatible_with},
     toolchain = "@{user_node_repository_name}_{platform}//:toolchain",
-    toolchain_type = "@rules_nodejs//nodejs:toolchain_type",
+    toolchain_type = "@rules_nodejs//nodejs:runtime_toolchain_type",
 )
 """.format(
             platform = platform,

nodejs/repositories.bzl

@@ -476,7 +476,7 @@ def nodejs_register_toolchains(name = DEFAULT_NODE_REPOSITORY, register = True,
         )
         if register:
             native.register_toolchains(
-                "@%s_toolchains//:%s_toolchain_target" % (name, platform),
+                "@%s_toolchains//:%s_runtime_toolchain" % (name, platform),
                 "@%s_toolchains//:%s_toolchain" % (name, platform),
             )
 

nodejs/semantics.bzl

@@ -0,0 +1,33 @@
+# Copyright 2021 The Bazel Authors. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Rules NodeJS Semantics"""
+
+_NODE_TOOLCHAIN_TYPE = Label("//nodejs:toolchain_type")
+_NODE_RUNTIME_TOOLCHAIN_TYPE = Label("//nodejs:runtime_toolchain_type")
+
+def _find_node_toolchain(ctx):
+    return ctx.toolchains[_NODE_TOOLCHAIN_TYPE].nodeinfo
+
+def _find_node_runtime_toolchain(ctx):
+    return ctx.toolchains[_NODE_RUNTIME_TOOLCHAIN_TYPE].nodeinfo
+
+semantics = struct(
+    NODE_TOOLCHAIN_LABEL = Label("//nodejs:current_node_toolchain"),
+    NODE_TOOLCHAIN_TYPE = _NODE_TOOLCHAIN_TYPE,
+    NODE_TOOLCHAIN = config_common.toolchain_type(_NODE_TOOLCHAIN_TYPE, mandatory = True),
+    find_node_toolchain = _find_node_toolchain,
+    NODE_RUNTIME_TOOLCHAIN_TYPE = _NODE_RUNTIME_TOOLCHAIN_TYPE,
+    NODE_RUNTIME_TOOLCHAIN = config_common.toolchain_type(_NODE_RUNTIME_TOOLCHAIN_TYPE, mandatory = True),
+    find_node_runtime_toolchain = _find_node_runtime_toolchain,
+)