Skip to content

Conversation

Andrewyx
Copy link

Hello! This PR provides extended support for projects which use pybind11_bazel to produce Python wheels.

As a developer on PyScreenReader, it was discovered that the current implementation of pybind11_bazel and its method of copying .so pybind_extension to .pyd file formats using copy_file does not work for the rules_python py_wheel rule. The reason for this appears to be that the generated .pyd file is not a true target which can be referred to by the py_wheel rule, or the dependency resolution graph struggles with this generated file strategy. This seems to have resulted in no one in the community developing Windows-based Python packages using pybind11 and Bazel.

Regardless, the solution here is to use a similar strategy mentioned in #74 which used a genrule to create a valid target for the aliasing.

@Mizux
Copy link
Collaborator

Mizux commented Sep 22, 2025

@Andrewyx
Copy link
Author

@Mizux Let me try this once I get home, I believe merging in that patch should work. However, I don't think the py_deps param is necessary since it implies that the pybinded extension itself requires some sort of Python dep (which is originally written in C++).

@Mizux
Copy link
Collaborator

Mizux commented Sep 22, 2025

IIRC It could be needed when your C++ pybind11 bind some methods and you want numpy support etc...

@Andrewyx
Copy link
Author

After some testing and fiddling, it looks like that patch does not work. The problem appears to be due to the py_library part of the macro:

    native.py_library(
        name = name,
        data = select({
            "@platforms//os:windows": [":" + name + ".pyd"],
            "//conditions:default": [":" + name + ".so"],
        }),
        deps = py_deps,
        visibility = visibility,
    )

Specifically, the data is not triggering the genrule to create the .pyd file, and in the finished wheel, the .pyd is not present (i.e. build of the wheel target succeeds, but the data itself is missing).

I like the idea of wrapping the macro using the py_library rule, but I think Bazel/pybind11 enforces strict semantic calls to particular data names to add them to the dependency tree (e.g. name + .so or name_ + .pyd). Using the exact names from the native cc_binary rules seems to cause the shared libs to generate. I suspect this is also where this stack overflow issue arises from. This also explains why using native.alias works, since it is directly calling the shared lib targets.

@Mizux Mizux self-assigned this Sep 26, 2025
@Mizux Mizux added bug Something isn't working enhancement New feature or request labels Sep 26, 2025
@Mizux
Copy link
Collaborator

Mizux commented Oct 1, 2025

@Andrewyx after doing few tests on a windows laptop it seems that:

=> leading to this PR:

@Andrewyx
Copy link
Author

Andrewyx commented Oct 1, 2025

Looks good and appears to be working on my end too! I'll go ahead and close this issue now. Thanks for the help! 😄

@Andrewyx Andrewyx closed this Oct 1, 2025
@Mizux Mizux added the OS: Windows Windows label Oct 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request OS: Windows Windows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants