-
Notifications
You must be signed in to change notification settings - Fork 23
Produce OS-specific wheels #1166
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 41 commits
Commits
Show all changes
56 commits
Select commit
Hold shift + click to select a range
552895b
Update releaser.yml to get and publish all .whl files from the Releas…
PProfizi 2f23846
Temp save of setup.py
PProfizi f1dbb61
Delete setup.py and declare everything in pyproject.toml
PProfizi 0d82262
Draft version of setup.cfg
PProfizi 28b51f7
Revert "Delete setup.py and declare everything in pyproject.toml"
PProfizi 01da8c1
Try setup.cfg with setup.py
PProfizi ab71aa9
Add wheel as build requirement in pyproject.toml
PProfizi 4f10ef7
Try build jobs in the CI
PProfizi 3a30034
Only run build jobs
PProfizi 4bdf849
Add build to requirements_build.txt
PProfizi 6678ec1
Update ci.yml build wheel job
PProfizi f29ac0e
Update ci.yml build wheel job
PProfizi 2825c7f
Create .ci/build_wheel.py script to create platform-specific wheels
PProfizi 17d007d
Update ci.yml to use .ci/build_wheel.py
PProfizi 039d1f6
Add --wheelhouse argument to .ci/build_wheel.py, switch to pip wheel
PProfizi ba5d6e8
Update ci.yml
PProfizi 82d0a1c
Update ci.yml
PProfizi 881cb8c
Update ci.Test build wheel during test
PProfizi cdad6e3
Skip build_package action in test.yml
PProfizi f01da43
Fix path to wheel
PProfizi b8e9c1d
Fix optional dependencies
PProfizi d9bf7e0
Upload wheel and test docstrings
PProfizi bc6c39d
Switch back to build when not building a wheelhouse
PProfizi dfec5c2
Test upload of wheelhouse
PProfizi 85df258
Do not add plotting optional dependencies when building the wheelhouse
PProfizi a55ab1f
Update wheelhouse name
PProfizi 9f6645d
Update wheelhouse name
PProfizi 0aa1c21
Update wheelhouse name
PProfizi aafe8ec
Set back wheelhouse arg to false in ci.yml/tests
PProfizi e743f82
Activate tests back in tests.yml
PProfizi 74d4ac3
Tests all tests in ci.yml
PProfizi 68cdc00
Update default python version in tests.yml
PProfizi bbe7ca0
Update examples.yml
PProfizi 13787bd
Update pydpf-post.yml
PProfizi b72acd1
Update docs.yml
PProfizi 9c5f579
Update test_docker.yml
PProfizi 0211458
Update examples_docker.yml
PProfizi 487e20a
Update examples_docker.yml
PProfizi 50078b8
Update gate.yml
PProfizi 30c4949
Update ci_release.yml
PProfizi 46778d3
Merge branch 'master' into ci/os_specific_wheels
PProfizi a94aaec
Add tests_any job
PProfizi 2cd3004
Add tests_any job
PProfizi eb90b5c
Update build_wheel.py to also remove the gatebin folder when any
PProfizi 81ac41b
Update build_wheel.py to also remove the gatebin folder when any
PProfizi d4bfeb1
Add timeout to tests.yml
PProfizi e934fd8
Remove build_any job from ci.yml
PProfizi 790b5ba
Add tests_any job to ci_release.yml
PProfizi 811ca3f
Merge branch 'master' into ci/os_specific_wheels
PProfizi 4909ac3
Fix test_load_api_without_awp_root_no_gatebin
PProfizi cbbaca5
Merge remote-tracking branch 'origin/ci/os_specific_wheels' into ci/o…
PProfizi 19cceb8
Fix test_load_api_without_awp_root_no_gatebin
PProfizi 09c479d
Merge branch 'master' into ci/os_specific_wheels
PProfizi 34bf6a0
Update test_service.py/test_load_api_without_awp_root_no_gatebin
PProfizi 5ed2b03
Remove gate.yml
PProfizi 723c0b4
Merge branch 'master' into ci/os_specific_wheels
PProfizi File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| # This script generates the different versions of the ansys-dpf-core wheels based on a given input. | ||
| # Input can be one of ["any", "win", "manylinux1", "manylinux_2_17"] | ||
|
|
||
| import argparse | ||
| import pathlib | ||
| import subprocess | ||
| import os | ||
| import sys | ||
| import shutil | ||
| import tempfile | ||
|
|
||
|
|
||
| supported_platforms = { | ||
| "any": "any", | ||
| "win": "win_amd64", | ||
| "manylinux1": "manylinux1_x86_64", | ||
| "manylinux_2_17": "manylinux_2_17_x86_64" | ||
| } | ||
|
|
||
| argParser = argparse.ArgumentParser() | ||
| argParser.add_argument("-p", "--platform", help="platform") | ||
| argParser.add_argument("-w", "--wheelhouse", help="platform", action='store_true') | ||
|
|
||
| args = argParser.parse_args() | ||
|
|
||
| if args.platform not in supported_platforms: | ||
| raise ValueError(f"Platform {args.platform} is not supported. " | ||
| f"Supported platforms are: {list(supported_platforms.keys())}") | ||
| else: | ||
| requested_platform = supported_platforms[args.platform] | ||
| print(requested_platform) | ||
|
|
||
| # Move binaries out of the source depending on platform requested | ||
| # any: move all binaries out before building | ||
| # win: move .so binaries out before building | ||
| # lin: move .dll binaries out before building | ||
| with tempfile.TemporaryDirectory() as tmpdirname: | ||
| print('Created temporary directory: ', tmpdirname) | ||
|
|
||
| # Create the temporary build-opts.cfg | ||
| build_opts_path = os.path.join(tmpdirname, "build-opts.cfg") | ||
| with open(build_opts_path, "w") as build_opts_file: | ||
| build_opts_file.write(f"[bdist_wheel]\nplat-name={requested_platform}") | ||
| os.environ["DIST_EXTRA_CONFIG"] = build_opts_path | ||
|
|
||
| # Move the binaries | ||
| gatebin_folder_path = os.path.join( | ||
| os.path.curdir, | ||
| os.path.join("src", "ansys", "dpf", "gatebin") | ||
| ) | ||
| binaries_to_move = [] | ||
| moved = [] | ||
| if "win" in requested_platform or "any" == requested_platform: | ||
| # Move linux binaries | ||
| binaries_to_move.extend(["libAns.Dpf.GrpcClient.so", "libDPFClientAPI.so"]) | ||
| if "linux" in requested_platform or "any" == requested_platform: | ||
| # Move windows binaries | ||
| binaries_to_move.extend(["Ans.Dpf.GrpcClient.dll", "DPFClientAPI.dll"]) | ||
|
|
||
| for binary_name in binaries_to_move: | ||
| src = os.path.join(gatebin_folder_path, binary_name) | ||
| dst = os.path.join(tmpdirname, binary_name) | ||
| print(f"Moving {src} to {dst}") | ||
| shutil.move(src=src, dst=dst) | ||
| moved.append([dst, src]) | ||
|
|
||
| # Call the build | ||
| if not args.wheelhouse: | ||
| cmd = [sys.executable, "-m", "build", "--wheel"] | ||
| else: | ||
| cmd = [sys.executable, "-m", "pip", "wheel", "-w", "dist", "."] | ||
| try: | ||
| subprocess.run(cmd, capture_output=False, text=True) | ||
| print("Done building the wheel.") | ||
| except Exception as e: | ||
| print(f"Build failed with error: {e}") | ||
|
|
||
| # Move binaries back | ||
| for move_back in moved: | ||
| print(f"Moving back {move_back[0]} to {move_back[1]}") | ||
| shutil.move(src=move_back[0], dst=move_back[1]) | ||
| print("Binaries moved back.") | ||
|
|
||
| print(f"Done building {requested_platform} wheel for ansys-dpf-core!") | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -66,24 +66,53 @@ jobs: | |
| echo "ANSYS_DPF_ACCEPT_LA=Y" >> $GITHUB_ENV | ||
| echo "ANSYSLMD_LICENSE_FILE=1055@${{ secrets.LICENSE_SERVER }}" >> $GITHUB_ENV | ||
|
|
||
| - name: Setup Python | ||
| uses: actions/[email protected].0 | ||
| - name: "Setup Python" | ||
| uses: actions/[email protected].1 | ||
| with: | ||
| python-version: ${{ inputs.python_version }} | ||
|
|
||
| - name: "Build Package" | ||
| id: build-package | ||
| uses: ansys/pydpf-actions/[email protected] | ||
| - name: "Install requirements" | ||
| run: pip install -r requirements/requirements_build.txt | ||
|
|
||
| - name: "Build the wheel" | ||
| shell: bash | ||
| run: | | ||
| if [ ${{ matrix.os }} == "ubuntu-latest" ]; then | ||
| export platform="manylinux_2_17" | ||
| else | ||
| export platform="win" | ||
| fi | ||
| python .ci/build_wheel.py -p $platform -w | ||
|
|
||
| - name: "Expose the wheel" | ||
| shell: bash | ||
| id: wheel | ||
| working-directory: dist | ||
| run: | | ||
| export name=`ls ansys_dpf_core*.whl` | ||
| echo ${name} | ||
| echo "wheel_name=${name[0]}" >> $GITHUB_OUTPUT | ||
|
|
||
| - name: "Install package wheel" | ||
| shell: bash | ||
| run: | | ||
| pip install dist/${{ steps.wheel.outputs.wheel_name }}[plotting] | ||
|
|
||
| - name: "Install DPF" | ||
| id: set-server-path | ||
| uses: ansys/pydpf-actions/[email protected] | ||
| with: | ||
| python-version: ${{ inputs.python_version }} | ||
| ANSYS_VERSION: ${{inputs.ANSYS_VERSION}} | ||
| PACKAGE_NAME: ${{env.PACKAGE_NAME}} | ||
| MODULE: ${{env.MODULE}} | ||
| dpf-standalone-TOKEN: ${{secrets.DPF_PIPELINE}} | ||
| install_extras: plotting | ||
| wheel: false | ||
| wheelhouse: false | ||
| standalone_suffix: ${{ inputs.standalone_suffix }} | ||
| ANSYS_VERSION : ${{inputs.ANSYS_VERSION}} | ||
|
|
||
| - name: "Check licences of packages" | ||
| uses: ansys/pydpf-actions/[email protected] | ||
|
|
||
| - name: "Test import" | ||
| shell: bash | ||
| working-directory: tests | ||
| run: python -c "from ansys.dpf import core" | ||
|
|
||
| - name: "Setup headless display" | ||
| uses: pyvista/setup-headless-display-action@v1 | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -71,23 +71,53 @@ jobs: | |
| echo "ANSYS_DPF_ACCEPT_LA=Y" >> $GITHUB_ENV | ||
| echo "ANSYSLMD_LICENSE_FILE=1055@${{ secrets.LICENSE_SERVER }}" >> $GITHUB_ENV | ||
|
|
||
| - name: Setup Python | ||
| - name: "Setup Python" | ||
| uses: actions/[email protected] | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
|
|
||
| - name: "Build Package" | ||
| uses: ansys/pydpf-actions/[email protected] | ||
| - name: "Install requirements" | ||
| run: pip install -r requirements/requirements_build.txt | ||
|
|
||
| - name: "Build the wheel" | ||
| shell: bash | ||
| run: | | ||
| if [ ${{ matrix.os }} == "ubuntu-latest" ]; then | ||
| export platform="manylinux_2_17" | ||
| else | ||
| export platform="win" | ||
| fi | ||
| python .ci/build_wheel.py -p $platform -w | ||
|
|
||
| - name: "Expose the wheel" | ||
| shell: bash | ||
| id: wheel | ||
| working-directory: dist | ||
| run: | | ||
| export name=`ls ansys_dpf_core*.whl` | ||
| echo ${name} | ||
| echo "wheel_name=${name[0]}" >> $GITHUB_OUTPUT | ||
|
|
||
| - name: "Install package wheel" | ||
| shell: bash | ||
| run: | | ||
| pip install dist/${{ steps.wheel.outputs.wheel_name }}[plotting] | ||
|
|
||
| - name: "Install DPF" | ||
| id: set-server-path | ||
| uses: ansys/pydpf-actions/[email protected] | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
| ANSYS_VERSION: ${{inputs.ANSYS_VERSION}} | ||
| PACKAGE_NAME: ${{ env.PACKAGE_NAME }} | ||
| MODULE: ${{ env.MODULE }} | ||
| dpf-standalone-TOKEN: ${{secrets.DPF_PIPELINE}} | ||
| install_extras: plotting | ||
| wheelhouse: false | ||
| wheel: false | ||
| standalone_suffix: ${{ inputs.standalone_suffix }} | ||
| ANSYS_VERSION : ${{inputs.ANSYS_VERSION}} | ||
|
|
||
| - name: "Check licences of packages" | ||
| uses: ansys/pydpf-actions/[email protected] | ||
|
|
||
| - name: "Test import" | ||
| shell: bash | ||
| working-directory: tests | ||
| run: python -c "from ansys.dpf import core" | ||
|
|
||
| - name: "Prepare Testing Environment" | ||
| uses: ansys/pydpf-actions/[email protected] | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if I remember correctly, wheels used to be build by dpf-action repo, what is done where now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what about pydpf-post?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@cbellot000 they indeed used to be created by an action in the dpf-action repo. Now I've changed to have the logic directly in the pydpf-core yml files. I will consider making a common action later but since this is particular to pydpf-core, it would not have a big advantage to move it to the pydpf-actions
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right now the pydpf-post regular CI just clones the pydpf-core repo and installs it as-is since the dependency is on the git repository directly. I'll have to make a PR on PyDPF-Post to properly build the ansys-dpf-core wheel based on the context, but it can be done after this.
The pydpf-post Release CI gets an ansys-dpf-core wheel from pip, so it requires no change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, will there be any time with a broken pipeline in pydpf-post? (it's just to make sure we capture evrything)
Also if this will be used in pydpf-post, I guess you'll move it to pydpf-actions after?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @cbellot000, sorry I do not understand the first sentence.
Edit: OK I think I can test running the pydpf-post pipelines on this branch to check what happens.
As for pydpf-post, indeed once I have a build_core action I can move it to the pydpf-actions, but this is really a consolidation step which is not really required at this point.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See here for a run of the PyDPF-Post CI while targeting this branch specifically in the requirements, meaning this branch will be cloned when pip installing dependencies of
ansys-dpf-post. Everything passes.