Skip to content

Conversation

@leo-amd
Copy link
Collaborator

@leo-amd leo-amd commented Dec 9, 2025

Description

This PR updates the CI pipeline to build and test TransformerEngine using official wheel packages instead of installing directly from the source.

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refactoring

Changes

  • Added a new build_wheels job to the workflow that builds TE wheels using Dockerfile.rocm.manylinux.x86].
  • Configured the workflow to upload generated wheels as GitHub Actions artifacts for temporary storage and downstream usage.
  • Modified the build_and_test job to download and mount the built wheel artifacts.
  • Updated the installation step in the test container to install TE from the generated .whl file instead of pip install . from source.

@leo-amd leo-amd changed the title CI: Decouple wheels building CI: Switch GHA pipeline to build and test wheels (Fixes #13333, #14193) Dec 9, 2025
@leo-amd leo-amd changed the title CI: Switch GHA pipeline to build and test wheels (Fixes #13333, #14193) CI: Switch GHA pipeline to build and test wheels Dec 9, 2025
@leo-amd leo-amd marked this pull request as draft December 9, 2025 15:55
pip install ninja
git config --global --add safe.directory '*'
pip install --no-build-isolation -v . 2>&1
pip install /wheelhouse_mount/transformer_engine*.whl --no-build-isolation --force-reinstall 2>&1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why force-reinstall is needed? Is TE already installed on this image ?

cd build_tools/wheel_utils
docker build -f Dockerfile.rocm.manylinux.x86 \
--build-arg ROCM_REPO_URL=https://repo.radeon.com/rocm/rhel8/latest/main/ \
-t te-builder .
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does te-builder need to be manually deleted and also, does docker cache keep anything between CI runs or it is always clean for every run?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants