-py_venv(name, data, deps, env, imports, interpreter_options, package_collisions, python_version, - resolutions, srcs) +VirtualenvInfo(home)-Build a Python virtual environment and execute its interpreter. -**ATTRIBUTES** + Provider used to distinguish venvs from py rules. + +**FIELDS** -| Name | Description | Type | Mandatory | Default | -| :------------- | :------------- | :------------- | :------------- | :------------- | -| name | A unique name for this target. | Name | required | | -| data | Runtime dependencies of the program.
data dependencies will be available in the .runfiles folder for this binary/test. The program may optionally use the Runfiles lookup library to locate the data files, see https://pypi.org/project/bazel-runfiles/. | List of labels | optional | [] |
-| deps | Targets that produce Python code, commonly py_library rules. | List of labels | optional | [] |
-| env | Environment variables to set when running the binary. | Dictionary: String -> String | optional | {} |
-| imports | List of import directories to be added to the PYTHONPATH. | List of strings | optional | [] |
-| interpreter_options | Additional options to pass to the Python interpreter. | List of strings | optional | [] |
-| package_collisions | The action that should be taken when a symlink collision is encountered when creating the venv. A collision can occur when multiple packages providing the same file are installed into the venv. The possible values are:"error" |
-| python_version | Whether to build this target and its transitive deps for a specific python version. | String | optional | "" |
-| resolutions | Satisfy a virtual_dep with a mapping from external package name to the label of an installed package that provides it. See [virtual dependencies](/docs/virtual_deps.md). | Dictionary: Label -> String | optional | {} |
-| srcs | Python source files. | List of labels | optional | [] |
+| Name | Description |
+| :------------- | :------------- |
+| home | Path of the virtualenv |
-
-## py_venv_binary
+
+
+## py_venv
-py_venv_binary(name, data, deps, env, imports, interpreter_options, main, package_collisions, - python_version, resolutions, srcs, venv) +py_venv(kwargs)-Run a Python program under Bazel using a virtualenv. -**ATTRIBUTES** +**PARAMETERS** -| Name | Description | Type | Mandatory | Default | -| :------------- | :------------- | :------------- | :------------- | :------------- | -| name | A unique name for this target. | Name | required | | -| data | Runtime dependencies of the program.
data dependencies will be available in the .runfiles folder for this binary/test. The program may optionally use the Runfiles lookup library to locate the data files, see https://pypi.org/project/bazel-runfiles/. | List of labels | optional | [] |
-| deps | Targets that produce Python code, commonly py_library rules. | List of labels | optional | [] |
-| env | Environment variables to set when running the binary. | Dictionary: String -> String | optional | {} |
-| imports | List of import directories to be added to the PYTHONPATH. | List of strings | optional | [] |
-| interpreter_options | Additional options to pass to the Python interpreter. | List of strings | optional | [] |
-| main | Script to execute with the Python interpreter. | Label | required | |
-| package_collisions | The action that should be taken when a symlink collision is encountered when creating the venv. A collision can occur when multiple packages providing the same file are installed into the venv. The possible values are:"error" |
-| python_version | Whether to build this target and its transitive deps for a specific python version. | String | optional | "" |
-| resolutions | Satisfy a virtual_dep with a mapping from external package name to the label of an installed package that provides it. See [virtual dependencies](/docs/virtual_deps.md). | Dictionary: Label -> String | optional | {} |
-| srcs | Python source files. | List of labels | optional | [] |
-| venv | A virtualenv; if provided all 3rdparty deps are assumed to come via the venv. | Label | optional | None |
+| Name | Description | Default Value |
+| :------------- | :------------- | :------------- |
+| kwargs | -
| none | - -## py_venv_test + + +## py_venv_binary-py_venv_test(name, data, deps, env, env_inherit, imports, interpreter_options, main, - package_collisions, python_version, resolutions, srcs, venv) +py_venv_binary(kwargs)-Run a Python program under Bazel using a virtualenv. -**ATTRIBUTES** +**PARAMETERS** -| Name | Description | Type | Mandatory | Default | -| :------------- | :------------- | :------------- | :------------- | :------------- | -| name | A unique name for this target. | Name | required | | -| data | Runtime dependencies of the program.
data dependencies will be available in the .runfiles folder for this binary/test. The program may optionally use the Runfiles lookup library to locate the data files, see https://pypi.org/project/bazel-runfiles/. | List of labels | optional | [] |
-| deps | Targets that produce Python code, commonly py_library rules. | List of labels | optional | [] |
-| env | Environment variables to set when running the binary. | Dictionary: String -> String | optional | {} |
-| env_inherit | Specifies additional environment variables to inherit from the external environment when the test is executed by bazel test. | List of strings | optional | [] |
-| imports | List of import directories to be added to the PYTHONPATH. | List of strings | optional | [] |
-| interpreter_options | Additional options to pass to the Python interpreter. | List of strings | optional | [] |
-| main | Script to execute with the Python interpreter. | Label | required | |
-| package_collisions | The action that should be taken when a symlink collision is encountered when creating the venv. A collision can occur when multiple packages providing the same file are installed into the venv. The possible values are:"error" |
-| python_version | Whether to build this target and its transitive deps for a specific python version. | String | optional | "" |
-| resolutions | Satisfy a virtual_dep with a mapping from external package name to the label of an installed package that provides it. See [virtual dependencies](/docs/virtual_deps.md). | Dictionary: Label -> String | optional | {} |
-| srcs | Python source files. | List of labels | optional | [] |
-| venv | A virtualenv; if provided all 3rdparty deps are assumed to come via the venv. | Label | optional | None |
+| Name | Description | Default Value |
+| :------------- | :------------- | :------------- |
+| kwargs | -
| none | - -## VirtualenvInfo + + +## py_venv_link-VirtualenvInfo(home) +py_venv_link(venv_name, kwargs)+Build a Python virtual environment and produce a script to link it into the build directory. - Provider used to distinguish venvs from py rules. - - -**FIELDS** +**PARAMETERS** -| Name | Description | -| :------------- | :------------- | -| home | Path of the virtualenv | +| Name | Description | Default Value | +| :------------- | :------------- | :------------- | +| venv_name |
-
|None |
+| kwargs | -
| none | - + -## py_venv_link +## py_venv_test-py_venv_link(venv_name, kwargs) +py_venv_test(kwargs)-Build a Python virtual environment and produce a script to link it into the build directory. + **PARAMETERS** | Name | Description | Default Value | | :------------- | :------------- | :------------- | -| venv_name |
-
|None |
-| kwargs | -
| none | +| kwargs |-
| none | diff --git a/py/private/py_venv/BUILD.bazel b/py/private/py_venv/BUILD.bazel index 2b46ae4c..d8953b88 100644 --- a/py/private/py_venv/BUILD.bazel +++ b/py/private/py_venv/BUILD.bazel @@ -10,6 +10,13 @@ exports_files([ "link.py", ]) +config_setting( + name = "debug_build", + values = { + "compilation_mode": "dbg", + }, +) + bzl_library( name = "py_venv", srcs = [ diff --git a/py/private/py_venv/entrypoint.tmpl.sh b/py/private/py_venv/entrypoint.tmpl.sh index 37ebfd67..69e47ea3 100644 --- a/py/private/py_venv/entrypoint.tmpl.sh +++ b/py/private/py_venv/entrypoint.tmpl.sh @@ -1,15 +1,15 @@ #!/usr/bin/env bash +if {{DEBUG}}; then + set -x +fi + {{BASH_RLOCATION_FN}} runfiles_export_envvars set -o errexit -o nounset -o pipefail -{{PRELUDE}} - source "$(rlocation "{{VENV}}")"/bin/activate -{{PREEXEC}} - exec "$(rlocation "{{VENV}}")"/bin/python {{INTERPRETER_FLAGS}} "$@" diff --git a/py/private/py_venv/py_venv.bzl b/py/private/py_venv/py_venv.bzl index ad1e8200..7315440c 100644 --- a/py/private/py_venv/py_venv.bzl +++ b/py/private/py_venv/py_venv.bzl @@ -36,30 +36,6 @@ def _interpreter_flags(ctx): return args -def _venv_preexec(ctx): - py_toolchain = _py_semantics.resolve_toolchain(ctx) - lines = [] - - if py_toolchain.runfiles_interpreter: - lines.extend([ - """\ -# HACK: Override PYTHONHOME after bin/activate to support embedded standalone interpreter -PYTHONHOME="$(dirname "$(dirname "$(rlocation {})")")" -export PYTHONHOME -""".format(to_rlocation_path(ctx, py_toolchain.python)), - """\ -# HACK: Shove the PYTHONHOME's bin/ _second_ on the path. -# First on the path will be VIRTUALENV/bin which we want to stay there. -# But we also need the interpreter's bin/ to be on the path so that it can be found. -IFS=: read -a _arr <<< "$PATH" -_arr=(\"${_arr[@]:0:1}\" \"${PYTHONHOME}/bin\" \"${_arr[@]:1}\") -_ifs=\"$IFS\"; IFS=:; PATH=\"${_arr[*]}\"; IFS=\"$_ifs\" -export PATH -""", - ]) - - return "\n".join(lines) - # FIXME: This is derived directly from the py_binary.bzl rule and should really # be a layer on top of it if we can figure out flowing the data around. This is # PoC quality. @@ -147,7 +123,8 @@ def _py_venv_base_impl(ctx): executable = venv_tool, arguments = [ "--location=" + venv_dir.path, - "--python=" + py_shim.bin.bin.path, + "--venv-shim=" + py_shim.bin.bin.path, + "--python=" + to_rlocation_path(ctx, py_toolchain.python) if py_toolchain.runfiles_interpreter else py_toolchain.python.path, "--pth-file=" + site_packages_pth_file.path, "--env-file=" + env_file.path, "--bin-dir=" + ctx.bin_dir.path, @@ -158,13 +135,13 @@ def _py_venv_base_impl(ctx): py_toolchain.interpreter_version_info.major, py_toolchain.interpreter_version_info.minor, ), - ], + ] + (["--debug"] if ctx.attr.debug else []), inputs = rfs.merge_all([ ctx.runfiles(files = [ site_packages_pth_file, env_file, venv_tool, - ]), + ] + py_toolchain.files.to_list()), py_shim.default_info.default_runfiles, ]).files, outputs = [ @@ -175,7 +152,8 @@ def _py_venv_base_impl(ctx): return venv_dir, rfs.merge_all([ ctx.runfiles(files = [ venv_dir, - ]), + ] + py_toolchain.files.to_list()), + ctx.attr._runfiles_lib[DefaultInfo].default_runfiles, ]) def _py_venv_rule_impl(ctx): @@ -193,9 +171,8 @@ def _py_venv_rule_impl(ctx): "{{BASH_RLOCATION_FN}}": BASH_RLOCATION_FUNCTION.strip(), "{{INTERPRETER_FLAGS}}": " ".join(_interpreter_flags(ctx)), "{{ENTRYPOINT}}": "${VIRTUAL_ENV}/bin/python", - "{{PRELUDE}}": "", - "{{PREEXEC}}": _venv_preexec(ctx), "{{VENV}}": to_rlocation_path(ctx, venv_dir), + "{{DEBUG}}": str(ctx.attr.debug).lower(), }, is_executable = True, ) @@ -209,9 +186,9 @@ def _py_venv_rule_impl(ctx): venv_dir, ]), executable = ctx.outputs.executable, - runfiles = rfs.merge(ctx.runfiles(files = [ - venv_dir, - ])), + runfiles = rfs.merge( + ctx.runfiles(files = [venv_dir]), + ), ), # FIXME: Does not provide PyInfo because venvs are supposed to be terminal artifacts. VirtualenvInfo( @@ -240,9 +217,6 @@ def _py_venv_binary_impl(ctx): py_toolchain.files, srcs_depset, ] + virtual_resolution.srcs + virtual_resolution.runfiles, - extra_runfiles_depsets = [ - ctx.attr._runfiles_lib[DefaultInfo].default_runfiles, - ], ) if not ctx.attr.venv: @@ -261,9 +235,8 @@ def _py_venv_binary_impl(ctx): substitutions = { "{{BASH_RLOCATION_FN}}": BASH_RLOCATION_FUNCTION.strip(), "{{INTERPRETER_FLAGS}}": " ".join(_interpreter_flags(ctx)), - "{{PRELUDE}}": "", - "{{PREEXEC}}": _venv_preexec(ctx), "{{VENV}}": to_rlocation_path(ctx, venv_dir), + "{{DEBUG}}": str(ctx.attr.debug).lower(), }, is_executable = True, ) @@ -320,6 +293,9 @@ A collision can occur when multiple packages providing the same file are install default = "//py/private/toolchain:resolved_venv_toolchain", cfg = "exec", ), + "debug": attr.bool( + default = False, + ), }) _attrs.update(**_py_library.attrs) @@ -370,7 +346,7 @@ py_venv_base = struct( cfg = python_version_transition, ) -py_venv = rule( +_py_venv = rule( doc = """Build a Python virtual environment and execute its interpreter.""", implementation = _py_venv_rule_impl, attrs = py_venv_base.attrs, @@ -379,17 +355,14 @@ py_venv = rule( cfg = py_venv_base.cfg, ) -def py_venv_link(venv_name = None, **kwargs): - """Build a Python virtual environment and produce a script to link it into the build directory.""" - link_script = str(Label("//py/private/py_venv:link.py")) - py_venv_binary( - args = [] + (["--venv-name=" + venv_name] if venv_name else []), - main = link_script, - srcs = [link_script], - **kwargs - ) +def _wrap_with_debug(rule): + def helper(**kwargs): + kwargs["debug"] = select({Label(":debug_build"): True, "//conditions:default": False}) + return rule(**kwargs) + + return helper -py_venv_binary = rule( +_py_venv_binary = rule( doc = """Run a Python program under Bazel using a virtualenv.""", implementation = _py_venv_binary_impl, attrs = py_venv_base.attrs | py_venv_base.binary_attrs, @@ -398,7 +371,7 @@ py_venv_binary = rule( cfg = py_venv_base.cfg, ) -py_venv_test = rule( +_py_venv_test = rule( doc = """Run a Python program under Bazel using a virtualenv.""", implementation = _py_venv_binary_impl, attrs = py_venv_base.attrs | py_venv_base.binary_attrs | py_venv_base.test_attrs, @@ -406,3 +379,20 @@ py_venv_test = rule( test = True, cfg = py_venv_base.cfg, ) + +py_venv = _wrap_with_debug(_py_venv) +py_venv_binary = _wrap_with_debug(_py_venv_binary) +py_venv_test = _wrap_with_debug(_py_venv_test) + +def py_venv_link(venv_name = None, **kwargs): + """Build a Python virtual environment and produce a script to link it into the build directory.""" + + # Note that the binary is already wrapped with debug + link_script = str(Label("//py/private/py_venv:link.py")) + kwargs["debug"] = select({Label(":debug_build"): True, "//conditions:default": False}) + py_venv_binary( + args = [] + (["--venv-name=" + venv_name] if venv_name else []), + main = link_script, + srcs = [link_script], + **kwargs + ) diff --git a/py/tests/py_venv_image_layer/my_app_amd64_layers_listing.yaml b/py/tests/py_venv_image_layer/my_app_amd64_layers_listing.yaml index 291df7ab..75826ff1 100644 --- a/py/tests/py_venv_image_layer/my_app_amd64_layers_listing.yaml +++ b/py/tests/py_venv_image_layer/my_app_amd64_layers_listing.yaml @@ -2495,7 +2495,7 @@ files: - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/ - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/ - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/ - - -rwxr-xr-x 0 0 0 1832 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin + - -rwxr-xr-x 0 0 0 1275 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/ - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/ - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/ @@ -2510,7 +2510,7 @@ files: - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/ - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/ - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/bin/ - - -rwxr-xr-x 0 0 0 1918 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/bin/activate + - -rwxr-xr-x 0 0 0 7827 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/bin/activate - -rwxr-xr-x 0 0 0 813320 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/bin/python - -rwxr-xr-x 0 0 0 813320 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/bin/python3 - -rwxr-xr-x 0 0 0 813320 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/bin/python3.9 @@ -2518,7 +2518,7 @@ files: - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/lib/python3.9/ - -rwxr-xr-x 0 0 0 323 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/pyvenv.cfg - -rwxr-xr-x 0 0 0 299 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/__main__.py - - -rwxr-xr-x 0 0 0 1832 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/my_app_bin + - -rwxr-xr-x 0 0 0 1275 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/my_app_bin - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/bazel_tools/ - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/bazel_tools/tools/ - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/bazel_tools/tools/bash/ diff --git a/py/tests/py_venv_image_layer/my_app_arm64_layers_listing.yaml b/py/tests/py_venv_image_layer/my_app_arm64_layers_listing.yaml index df617a8b..1d062521 100644 --- a/py/tests/py_venv_image_layer/my_app_arm64_layers_listing.yaml +++ b/py/tests/py_venv_image_layer/my_app_arm64_layers_listing.yaml @@ -2476,7 +2476,7 @@ files: - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/ - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/ - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/ - - -rwxr-xr-x 0 0 0 1833 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin + - -rwxr-xr-x 0 0 0 1275 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/ - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/ - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/ @@ -2491,7 +2491,7 @@ files: - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/ - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/ - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/bin/ - - -rwxr-xr-x 0 0 0 1918 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/bin/activate + - -rwxr-xr-x 0 0 0 7828 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/bin/activate - -rwxr-xr-x 0 0 0 693968 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/bin/python - -rwxr-xr-x 0 0 0 693968 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/bin/python3 - -rwxr-xr-x 0 0 0 693968 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/bin/python3.9 @@ -2499,7 +2499,7 @@ files: - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/lib/python3.9/ - -rwxr-xr-x 0 0 0 323 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/.my_app_bin/pyvenv.cfg - -rwxr-xr-x 0 0 0 299 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/__main__.py - - -rwxr-xr-x 0 0 0 1833 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/my_app_bin + - -rwxr-xr-x 0 0 0 1275 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/aspect_rules_py/py/tests/py_venv_image_layer/my_app_bin - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/bazel_tools/ - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/bazel_tools/tools/ - drwxr-xr-x 0 0 0 0 Jan 1 2023 ./py/tests/py_venv_image_layer/my_app_bin.runfiles/bazel_tools/tools/bash/ diff --git a/py/tools/py/BUILD.bazel b/py/tools/py/BUILD.bazel index e7a60161..adaefd99 100644 --- a/py/tools/py/BUILD.bazel +++ b/py/tools/py/BUILD.bazel @@ -12,6 +12,7 @@ rust_library( "src/_virtualenv.py", "src/activate.tmpl", "src/pyvenv.cfg.tmpl", + "src/runfiles_interpreter.tmpl", ], visibility = [ "//py/tools/unpack_bin:__pkg__", diff --git a/py/tools/py/src/activate.tmpl b/py/tools/py/src/activate.tmpl index 818366bf..acf417b4 100644 --- a/py/tools/py/src/activate.tmpl +++ b/py/tools/py/src/activate.tmpl @@ -1,63 +1,89 @@ # Adapted from CPython Lib/venv/scripts/common/activate -# -# This file must be used with "source bin/activate" *from bash* -# You cannot run it directly + +set -eu -o pipefail deactivate () { # reset old environment variables if [ -n "${_OLD_VIRTUAL_PATH:-}" ] ; then PATH="${_OLD_VIRTUAL_PATH:-}" export PATH - unset _OLD_VIRTUAL_PATH fi - if [ -n "${_OLD_VIRTUAL_PYTHONHOME:-}" ] ; then + + if [ "${_OLD_VIRTUAL_PYTHONHOME:-}" = "_activate_undef" ]; then + unset _OLD_VIRTUAL_PYTHONHOME + unset PYTHONHOME + elif [ -n "${_OLD_VIRTUAL_PYTHONHOME:-}" ] ; then PYTHONHOME="${_OLD_VIRTUAL_PYTHONHOME:-}" export PYTHONHOME - unset _OLD_VIRTUAL_PYTHONHOME fi - # Call hash to forget past locations. Without forgetting - # past locations the $PATH changes we made may not be respected. - # See "man bash" for more details. hash is usually a builtin of your shell + # Call hash to forget past locations. Without forgetting past locations the + # $PATH changes we made may not be respected. See "man bash" for more + # details. hash is usually a builtin of your shell hash -r 2> /dev/null if [ -n "${_OLD_VIRTUAL_PS1:-}" ] ; then PS1="${_OLD_VIRTUAL_PS1:-}" export PS1 - unset _OLD_VIRTUAL_PS1 fi + unset _OLD_VIRTUAL_PS1 + unset _OLD_VIRTUAL_PATH + unset _OLD_VIRTUAL_PYTHONHOME unset VIRTUAL_ENV unset VIRTUAL_ENV_PROMPT + + # Unset Bazel-injected vars +{{ENVVARS_UNSET}} + + # Unset vars we set with the runfiles interpreter. Note that this needs to + # be conditional so we don't throw this state out under tests or run. + if [ "${_OLD_RUNFILES_DIR:-}" = "_activate_undef" ]; then + unset RUNFILES_DIR + unset RUNFILES_MANIFEST_FILE + fi + if [ ! "${1:-}" = "nondestructive" ] ; then # Self destruct! unset -f deactivate fi } +{{DEBUG}} + # unset irrelevant variables deactivate nondestructive -# use the path as-is -# HACK: $BASH_SOURCE works on real bashes, $0 works on zsh -VIRTUAL_ENV="$(realpath "$(dirname "$(dirname "${BASH_SOURCE-$0}")")")" +# For ZSH, emulate BASH_SOURCE. +# The runfiles library code has some deps on this so we just set it :/ +: "${BASH_SOURCE:=$0}" + +VIRTUAL_ENV="$(realpath "$(dirname "$(dirname "${BASH_SOURCE}")")")" export VIRTUAL_ENV +# unset PYTHONHOME if set +# this will fail if PYTHONHOME is set to the empty string (which is bad anyway) +# could use `if (set -u; : $PYTHONHOME) ;` in bash. +_OLD_VIRTUAL_PYTHONHOME="${PYTHONHOME:-}" +unset PYTHONHOME + _OLD_VIRTUAL_PATH="$PATH" + +# Aspect additions +# We set these before runfiles initialization so that we can use it as part of a fallback path +{{ENVVARS}} + +# Initialize the runfiles interpreter if we're using one. Note that this happens +# AFTER unsetting the PYTHONHOME so that we can set PYTHONHOME if we're using a +# full bundled interpreter, and after we set the Bazel-specific envvars so we +# can provide some fallback handling around runfiles too. +{{RUNFILES_INTERPRETER}} + PATH="$VIRTUAL_ENV/bin:$PATH" export PATH -# unset PYTHONHOME if set -# this will fail if PYTHONHOME is set to the empty string (which is bad anyway) -# could use `if (set -u; : $PYTHONHOME) ;` in bash -if [ -n "${PYTHONHOME:-}" ] ; then - _OLD_VIRTUAL_PYTHONHOME="${PYTHONHOME:-}" - unset PYTHONHOME -fi - # Call hash to forget past commands. Without forgetting # past commands the $PATH changes we made may not be respected hash -r 2> /dev/null -# Aspect additions -{{ENVVARS}} +set +eu +o pipefail diff --git a/py/tools/py/src/runfiles_interpreter.tmpl b/py/tools/py/src/runfiles_interpreter.tmpl new file mode 100644 index 00000000..a4e3dc2a --- /dev/null +++ b/py/tools/py/src/runfiles_interpreter.tmpl @@ -0,0 +1,111 @@ +# --- Runfiles-based interpreter setup -- + +# If the runfiles dir is unset AND we will fail to find a runfiles manifest +# based on inspecting $0, we need to try something different. +# +# What this means is that: +# +# - This script isn't being loaded from `bazel run` +# +# - The script is likely being loaded directly as "activate" rather than via a +# launcher binary in which case the manifest would be obvious +# +# So we need to try and find the runfiles manifest by other means. + +_activate_find_runfiles() { + # $1 -- an executable path + if [[ "${1}" == */execroot/*/bin/* ]]; then + # Examples: + # - ${BAZEL_HOME}/execroot/aspect_rules_py/bazel-out/darwin_arm64-fastbuild/bin/ + # + # We can grab the execroot prefix, and then use the Bazel target info to + # find the manifest file and runfiles tree relative to the execroot. + + # HACK: We can't lazy-match to the first /bin/, so we have to manually count four groups + EXECROOT="$(echo "${1}" | sed 's/\(execroot\/[^\/]*\/[^\/]*\/[^\/]*\/[^\/]*\/\).*$/\1/' )" + export RUNFILES_DIR="${EXECROOT}/${RUNFILES_PATH}" + elif [[ "${1}" == *.runfiles/* ]]; then + # Examples: + # - bazel-bin/examples/py_venv/internal_venv.runfiles/aspect_rules_py/examples/py_venv/.internal_venv/bin/activate + # + # We are within the runfiles tree, so we just need to capture its root + export RUNFILES_DIR="$(echo "${1}" | sed 's/\(.runfiles\).*$/\1/')" + else + return 1 + fi +} + +if [ -z "${RUNFILES_DIR:-}" ] && \ + [ -z "${RUNFILES_MANIFEST_FILE:-}" ] && \ + [ ! -e "${BASH_SOURCE:-}.runfiles" ] && \ + [ ! -e "${BASH_SOURCE:-}.runfiles_manifest" ]; then + + # There are two cases here. + # 1. In development, the realpath will be in a /execroot/ somewhere + # 2. If copied to "production", the realpath will be in a .runfiles/ somewhere + + RUNFILES_PATH="$(echo "${BAZEL_TARGET}" | sed 's/^.*\/\/\(.*\):\(.*\)$/\1\/\2/' ).runfiles" + + set -uo pipefail; + _activate_find_runfiles "${BASH_SOURCE}" || \ + _activate_find_runfiles "$(realpath "${BASH_SOURCE}")" || \ + { echo>&2 "ERROR: activate[.sh] cannot identify a fallback runfiles manifest!"; exit 1; }; + + # FIXME: This should always be true, when is it not? + if [ -e "${RUNFILES_DIR}/MANIFEST" ]; then + RUNFILES_MANIFEST_FILE="${RUNFILES_DIR}/MANIFEST" + export RUNFILES_MANIFEST_FILE + fi + + # Set our magic flag to unset the runfiles vars + _OLD_RUNFILES_DIR="_activate_undef" +fi + +# As a workaround for export -f under zsh, we fence this whole thing off and pipe it to /dev/null +# HACK: Note that this is adjusted to use $BASH_SOURCE not $0; this works around other zsh vs bash issues +{ + +# --- begin runfiles.bash initialization v3 --- +# Copy-pasted from the Bazel Bash runfiles library v3. +# https://github.com/bazelbuild/bazel/blob/master/tools/bash/runfiles/runfiles.bash +set -uo pipefail; set +e; f=bazel_tools/tools/bash/runfiles/runfiles.bash +source "${RUNFILES_DIR:-/dev/null}/${f}" 2>/dev/null || \ + source "$(grep -sm1 "^${f} " "${RUNFILES_MANIFEST_FILE:-/dev/null}" | cut -f2- -d' ')" 2>/dev/null || \ + source "${BASH_SOURCE}.runfiles/${f}" 2>/dev/null || \ + source "$(grep -sm1 "^${f} " "${BASH_SOURCE}.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + source "$(grep -sm1 "^${f} " "${BASH_SOURCE}.exe.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + { echo>&2 "ERROR: runfiles.bash initializer cannot find ${f}. An executable rule may have forgotten to expose it in the runfiles, or the binary may require RUNFILES_DIR to be set."; exit 1; }; f=; set -e +# --- end runfiles.bash initialization v3 --- + +} >/dev/null + +# Look up the runfiles-based interpreter and put its dir _first_ on the path. +INTERPRETER="$(realpath $(rlocation {{INTERPRETER_TARGET}}))" + +# Figure out if we're dealing with just some program or a real install +#