-
-
Notifications
You must be signed in to change notification settings - Fork 11
CI: Adding and testing Big-Endian support for quaddtype #152
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
61406c5
init big endian ci
SwayamInSync e1ccd4b
Merge branch 'numpy:main' into big-endian-ci
SwayamInSync 977518e
inspiration from numpy test-1
SwayamInSync 33ed4c1
inspiration from numpy test-1
SwayamInSync 09764a5
onlynig-endian
SwayamInSync 77fcb35
using python -m
SwayamInSync 96be1d7
install pytest man
SwayamInSync c881907
Fix test directory path in big_endian.yml
SwayamInSync 68793e6
building and testing in one
SwayamInSync 814fafb
fixing endianes detection
SwayamInSync d9ae033
switching to slee_q
SwayamInSync 96e60fa
lets verify macros are working or not
SwayamInSync d777529
it was dragon4 all along
SwayamInSync 2d42806
win fix
SwayamInSync 19231fc
Merge branch 'main' into big-endian-ci
SwayamInSync File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,142 @@ | ||
name: Big-Endian Architecture Tests | ||
|
||
on: | ||
pull_request: | ||
branches: | ||
- main | ||
paths: | ||
- "quaddtype/**" | ||
- ".github/workflows/**" | ||
workflow_dispatch: | ||
|
||
defaults: | ||
run: | ||
shell: bash | ||
|
||
concurrency: | ||
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }} | ||
cancel-in-progress: true | ||
|
||
permissions: | ||
contents: read | ||
|
||
jobs: | ||
big_endian_tests: | ||
runs-on: ubuntu-22.04 | ||
continue-on-error: true | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
BUILD_PROP: | ||
- [ | ||
"s390x (IBM Z Big Endian)", | ||
"s390x-linux-gnu", | ||
"s390x/ubuntu:22.04", | ||
"s390x", | ||
] | ||
- [ | ||
"s390x - baseline(Z13)", | ||
"s390x-linux-gnu", | ||
"s390x/ubuntu:22.04", | ||
"s390x", | ||
] | ||
env: | ||
ARCH_NAME: ${{ matrix.BUILD_PROP[0] }} | ||
TOOLCHAIN_NAME: ${{ matrix.BUILD_PROP[1] }} | ||
DOCKER_CONTAINER: ${{ matrix.BUILD_PROP[2] }} | ||
ARCH: ${{ matrix.BUILD_PROP[3] }} | ||
TERM: xterm-256color | ||
|
||
name: "${{ matrix.BUILD_PROP[0] }}" | ||
steps: | ||
- uses: actions/checkout@v4 | ||
with: | ||
submodules: recursive | ||
fetch-tags: true | ||
persist-credentials: false | ||
|
||
- name: Initialize binfmt_misc for qemu-user-static | ||
run: | | ||
# Enable QEMU user-mode emulation for cross-architecture execution | ||
docker run --rm --privileged tonistiigi/binfmt:qemu-v9.2.2-52 --install all | ||
- name: Install cross-compilation toolchain | ||
run: | | ||
sudo apt update | ||
sudo apt install -y ninja-build gcc-${TOOLCHAIN_NAME} g++-${TOOLCHAIN_NAME} gfortran-${TOOLCHAIN_NAME} | ||
- name: Cache docker container | ||
uses: actions/cache@v4 | ||
id: container-cache | ||
with: | ||
path: ~/docker_${{ matrix.BUILD_PROP[1] }} | ||
key: container-quaddtype-${{ runner.os }}-${{ matrix.BUILD_PROP[1] }}-${{ matrix.BUILD_PROP[2] }}-${{ hashFiles('quaddtype/pyproject.toml') }} | ||
|
||
- name: Create cross-compilation container | ||
if: steps.container-cache.outputs.cache-hit != 'true' | ||
run: | | ||
docker run --platform=linux/${ARCH} --name quaddtype_container --interactive \ | ||
-v /:/host -v $(pwd):/workspace ${DOCKER_CONTAINER} /bin/bash -c " | ||
# Update package manager and install essential tools | ||
apt update && | ||
apt install -y cmake git python3 python-is-python3 python3-dev python3-pip build-essential && | ||
# Create necessary symlinks for cross-compilation | ||
mkdir -p /lib64 && ln -sf /host/lib64/ld-* /lib64/ || true && | ||
ln -sf /host/lib/x86_64-linux-gnu /lib/x86_64-linux-gnu || true && | ||
# Link cross-compilation toolchain from host | ||
rm -rf /usr/${TOOLCHAIN_NAME} && ln -sf /host/usr/${TOOLCHAIN_NAME} /usr/${TOOLCHAIN_NAME} && | ||
rm -rf /usr/lib/gcc/${TOOLCHAIN_NAME} && ln -sf /host/usr/lib/gcc-cross/${TOOLCHAIN_NAME} /usr/lib/gcc/${TOOLCHAIN_NAME} && | ||
# Set up compiler symlinks | ||
rm -f /usr/bin/gcc && ln -sf /host/usr/bin/${TOOLCHAIN_NAME}-gcc /usr/bin/gcc && | ||
rm -f /usr/bin/g++ && ln -sf /host/usr/bin/${TOOLCHAIN_NAME}-g++ /usr/bin/g++ && | ||
rm -f /usr/bin/gfortran && ln -sf /host/usr/bin/${TOOLCHAIN_NAME}-gfortran /usr/bin/gfortran && | ||
# Set up binutils | ||
rm -f /usr/bin/ar && ln -sf /host/usr/bin/${TOOLCHAIN_NAME}-ar /usr/bin/ar && | ||
rm -f /usr/bin/as && ln -sf /host/usr/bin/${TOOLCHAIN_NAME}-as /usr/bin/as && | ||
rm -f /usr/bin/ld && ln -sf /host/usr/bin/${TOOLCHAIN_NAME}-ld /usr/bin/ld && | ||
rm -f /usr/bin/ld.bfd && ln -sf /host/usr/bin/${TOOLCHAIN_NAME}-ld.bfd /usr/bin/ld.bfd && | ||
# Link build tools | ||
rm -f /usr/bin/ninja && ln -sf /host/usr/bin/ninja /usr/bin/ninja && | ||
rm -f /usr/local/bin/ninja && mkdir -p /usr/local/bin && ln -sf /host/usr/bin/ninja /usr/local/bin/ninja && | ||
# Configure git for workspace access | ||
git config --global --add safe.directory /workspace && | ||
# Install Python build dependencies | ||
python -m pip install --upgrade pip && | ||
python -m pip install meson>=1.3.2 meson-python wheel numpy && | ||
python -m pip install pytest pytest-run-parallel pytest-timeout && | ||
# Install system dependencies for quaddtype (SLEEF dependencies) | ||
apt install -y libssl-dev libfftw3-dev pkg-config | ||
" | ||
docker commit quaddtype_container quaddtype_container | ||
mkdir -p "~/docker_${TOOLCHAIN_NAME}" | ||
docker save -o "~/docker_${TOOLCHAIN_NAME}/quaddtype_container.tar" quaddtype_container | ||
- name: Load container from cache | ||
if: steps.container-cache.outputs.cache-hit == 'true' | ||
run: docker load -i "~/docker_${TOOLCHAIN_NAME}/quaddtype_container.tar" | ||
|
||
- name: Build quaddtype with cross-compilation and testing | ||
run: | | ||
docker run --rm --platform=linux/${ARCH} -e "TERM=xterm-256color" \ | ||
-v $(pwd):/workspace -v /:/host quaddtype_container \ | ||
/bin/script -e -q -c "/bin/bash --noprofile --norc -eo pipefail -c ' | ||
cd /workspace/quaddtype && | ||
echo \"Building quaddtype for ${ARCH_NAME}...\" && | ||
# Set OpenMP linking for cross-compilation | ||
export LDFLAGS=\"-fopenmp\" && | ||
# Install quaddtype with test dependencies | ||
python -m pip install .[test] -v --no-build-isolation --force-reinstall | ||
cd .. | ||
python -m pytest -vvv --color=yes --timeout=600 --tb=short quaddtype/tests/ | ||
'" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -1885,8 +1885,13 @@ Dragon4_PrintFloat_Sleef_quad(Sleef_quad *value, Dragon4_Options *opt) | |
union { | ||
Sleef_quad q; | ||
struct { | ||
#if defined(__BYTE_ORDER__) && (__BYTE_ORDER__ == __ORDER_BIG_ENDIAN__) | ||
npy_uint64 hi; | ||
npy_uint64 lo; | ||
#else | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nice! glad you chased this down. While BigEndian machines are rare these days it's nice to have fully correct code and it's important for it to be reflected in the source code that this representation is endian-dependent. |
||
npy_uint64 lo; | ||
npy_uint64 hi; | ||
#endif | ||
} i; | ||
} u; | ||
u.q = *value; | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This step takes 10 minutes. It might be possible to add some caching to make that quicker. Just a note for a future improvement.