Skip to content

Optimize and Enable Runtime Datascience Images for IBM Z[s390x] (Python 3.11 & 3.12) #1513

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
+520 −278
Open

Optimize and Enable Runtime Datascience Images for IBM Z[s390x] (Python 3.11 & 3.12) #1513

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
12c97e9
Enable s390x support for runtime-datascience (Python 3.11 & 3.12)
Nash-123 Jul 25, 2025
c589816
Update Pipfile.lock for Python 3.12
github-actions[bot] Aug 8, 2025
5108080
Update Pipfile.lock for Python 3.11
github-actions[bot] Aug 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions ci/cached-builds/gen_gha_matrix_jobs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@
"runtime-minimal-ubi9-python-3.12",
"jupyter-minimal-ubi9-python-3.11",
"jupyter-minimal-ubi9-python-3.12",
"runtime-datascience-ubi9-python-3.11",
"runtime-datascience-ubi9-python-3.12",
# add more here
}

Expand Down
82 changes: 41 additions & 41 deletions jupyter/pytorch+llmcompressor/ubi9-python-3.11/Pipfile.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

44 changes: 22 additions & 22 deletions jupyter/pytorch+llmcompressor/ubi9-python-3.11/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -896,9 +896,9 @@ httpcore==1.0.9; python_version >= '3.8' \
httpx==0.28.1; python_version >= '3.8' \
--hash=sha256:75e98c5f16b0f35b567856f597f06ff2270a374470a5c2392242528e3e3e42fc \
--hash=sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad
huggingface-hub==0.34.3; python_full_version >= '3.8.0' \
--hash=sha256:5444550099e2d86e68b2898b09e85878fbd788fc2957b506c6a79ce060e39492 \
--hash=sha256:d58130fd5aa7408480681475491c0abd7e835442082fbc3ef4d45b6c39f83853
huggingface-hub==0.34.4; python_full_version >= '3.8.0' \
--hash=sha256:9b365d781739c93ff90c359844221beef048403f1bc1f1c123c191257c3c890a \
--hash=sha256:a4228daa6fb001be3f4f4bdaf9a0db00e1739235702848df00885c9b5742c85c
idna==3.10; python_version >= '3.6' \
--hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \
--hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3
Expand Down Expand Up @@ -2871,22 +2871,22 @@ s3transfer==0.11.5; python_version >= '3.8' \
sacrebleu==2.5.1; python_version >= '3.8' \
--hash=sha256:1a088cc1c74ffaff0759c3191a85db09eecfa7a52e09be244e319d8d64e2fb11 \
--hash=sha256:7c9f7ee75bec3a5bf19dd87112dfd654952130e403ad30c48298fb7da3212d5d
safetensors==0.6.1; python_version >= '3.9' \
--hash=sha256:01b51af8cb7a3870203f2735e3c7c24d1a65fb2846e75613c8cf9d284271eccc \
--hash=sha256:1b62eab84e2c69918b598272504c5d2ebfe64da6c16fdf8682054eec9572534d \
--hash=sha256:294040ff20ebe079a2b4976cfa9a5be0202f56ca4f7f190b4e52009e8c026ceb \
--hash=sha256:4720957052d57c5ac48912c3f6e07e9a334d9632758c9b0c054afba477fcbe2d \
--hash=sha256:5dd969a01c738104f707fa0e306b757f5beb3ebdcd682fe0724170a0bf1c21fb \
--hash=sha256:64a733886d79e726899b9d9643813e48a2eec49f3ef0fdb8cd4b8152046101c3 \
--hash=sha256:6f16289e2af54affd591dd78ed12b5465e4dc5823f818beaeddd49a010cf3ba7 \
--hash=sha256:75693208b492a026b926edeebbae888cc644433bee4993573ead2dc44810b519 \
--hash=sha256:7c3d8d34d01673d1a917445c9437ee73a9d48bc6af10352b84bbd46c5da93ca5 \
--hash=sha256:81ed1b69d6f8acd7e759a71197ce3a69da4b7e9faa9dbb005eb06a83b1a4e52d \
--hash=sha256:a766ba6e19b198eff09be05f24cd89eda1670ed404ae828e2aa3fc09816ba8d8 \
--hash=sha256:a8687b71ac67a0b3f8ce87df9e8024edf087e94c34ef46eaaad694dce8d2f83f \
--hash=sha256:d498363746555dccffc02a47dfe1dee70f7784f3f37f1d66b408366c5d3a989e \
--hash=sha256:eed2079dca3ca948d7b0d7120396e776bbc6680637cf199d393e157fde25c937 \
--hash=sha256:f233dc3b12fb641b36724844754b6bb41349615a0e258087560968d6da92add5
safetensors==0.6.2; python_version >= '3.9' \
--hash=sha256:1d2d2b3ce1e2509c68932ca03ab8f20570920cd9754b05063d4368ee52833ecd \
--hash=sha256:43ff2aa0e6fa2dc3ea5524ac7ad93a9839256b8703761e76e2d0b2a3fa4f15d9 \
--hash=sha256:8045db2c872db8f4cbe3faa0495932d89c38c899c603f21e9b6486951a5ecb8f \
--hash=sha256:81e67e8bab9878bb568cffbc5f5e655adb38d2418351dc0859ccac158f753e19 \
--hash=sha256:89a89b505f335640f9120fac65ddeb83e40f1fd081cb8ed88b505bdccec8d0a1 \
--hash=sha256:93de35a18f46b0f5a6a1f9e26d91b442094f2df02e9fd7acf224cfec4238821a \
--hash=sha256:9c85ede8ec58f120bad982ec47746981e210492a6db876882aa021446af8ffba \
--hash=sha256:b0e4d029ab0a0e0e4fdf142b194514695b1d7d3735503ba700cf36d0fc7136ce \
--hash=sha256:c7b214870df923cbc1593c3faee16bec59ea462758699bd3fee399d00aac072c \
--hash=sha256:cab75ca7c064d3911411461151cb69380c9225798a20e712b102edda2542ddb1 \
--hash=sha256:d6675cf4b39c98dbd7d940598028f3742e0375a6b4d4277e76beb0c35f4b843b \
--hash=sha256:d83c20c12c2d2f465997c51b7ecb00e407e5f94d7dec3ea0cc11d86f60d3fde5 \
--hash=sha256:d944cea65fad0ead848b6ec2c37cc0b197194bec228f8020054742190e9312ac \
--hash=sha256:fa48268185c52bfe8771e46325a1e21d317207bcabcb72e65c6e28e9ffeb29c7 \
--hash=sha256:fc4d0d0b937e04bdf2ae6f70cd3ad51328635fe0e6214aa1fc811f3b576b3bda
Comment on lines +2874 to +2889
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

safetensors 0.6.2: confirm s390x wheel availability or gate for s390x

If this image is ever built for s390x, safetensors may require Rust if wheels aren’t available. Consider gating on s390x to avoid unexpected source builds, or plan a Rust toolchain stage.

Would you verify s390x wheel availability for safetensors 0.6.2?

If gating is desired, adjust the marker:

-safetensors==0.6.2; python_version >= '3.9' \
+safetensors==0.6.2; python_version >= '3.9' and platform_machine != 's390x' \

🌐 Web query:

Do PyPI wheels for safetensors==0.6.2 include linux s390x for Python 3.11?

💡 Result:

As of August 8, 2025, the PyPI wheels for safetensors version 0.6.2 do not include a pre-built wheel for the linux s390x architecture with Python 3.11. The available wheels for this version are:

  • cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
  • cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
  • cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
  • cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
  • cp311-cp311-manylinux_2_17_i686.manylinux1_i686.whl
  • cp311-cp311-macosx_11_0_arm64.whl
  • cp311-cp311-macosx_10_12_x86_64.whl

These wheels are compatible with various architectures, including x86_64, aarch64, armv7l, ppc64le, i686, and macOS platforms.

For the linux s390x architecture, the most recent available wheel is for safetensors version 0.5.3, which includes:

  • cp38-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl

This wheel is compatible with Python 3.8. (pypi.org)

Given the absence of a pre-built wheel for safetensors 0.6.2 on linux s390x with Python 3.11, you may need to build the package from source. Ensure that you have the necessary build tools and dependencies installed. Detailed instructions for building safetensors from source can be found in the project's documentation.

Citations:

safetensors 0.6.2 lacks linux/s390x wheel—gate or add Rust toolchain

safetensors 0.6.2 publishes wheels for x86_64, aarch64, armv7l, ppc64le, i686 and macOS, but no Python 3.11 manylinux wheel for s390x. On s390x this will fall back to source build (requiring Rust). To prevent unexpected Rust builds, either gate out s390x installs or ensure a Rust/Cargo stage is present.

Locations to address:

  • jupyter/pytorch+llmcompressor/ubi9-python-3.11/requirements.txt (lines 2874–2889)

Suggested gating diff:

-safetensors==0.6.2; python_version >= '3.9' \
+safetensors==0.6.2; python_version >= '3.9' and platform_machine != 's390x' \

Alternatively, update your s390x image builds to install the Rust toolchain before pip.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
safetensors==0.6.2; python_version >= '3.9' \
--hash=sha256:1d2d2b3ce1e2509c68932ca03ab8f20570920cd9754b05063d4368ee52833ecd \
--hash=sha256:43ff2aa0e6fa2dc3ea5524ac7ad93a9839256b8703761e76e2d0b2a3fa4f15d9 \
--hash=sha256:8045db2c872db8f4cbe3faa0495932d89c38c899c603f21e9b6486951a5ecb8f \
--hash=sha256:81e67e8bab9878bb568cffbc5f5e655adb38d2418351dc0859ccac158f753e19 \
--hash=sha256:89a89b505f335640f9120fac65ddeb83e40f1fd081cb8ed88b505bdccec8d0a1 \
--hash=sha256:93de35a18f46b0f5a6a1f9e26d91b442094f2df02e9fd7acf224cfec4238821a \
--hash=sha256:9c85ede8ec58f120bad982ec47746981e210492a6db876882aa021446af8ffba \
--hash=sha256:b0e4d029ab0a0e0e4fdf142b194514695b1d7d3735503ba700cf36d0fc7136ce \
--hash=sha256:c7b214870df923cbc1593c3faee16bec59ea462758699bd3fee399d00aac072c \
--hash=sha256:cab75ca7c064d3911411461151cb69380c9225798a20e712b102edda2542ddb1 \
--hash=sha256:d6675cf4b39c98dbd7d940598028f3742e0375a6b4d4277e76beb0c35f4b843b \
--hash=sha256:d83c20c12c2d2f465997c51b7ecb00e407e5f94d7dec3ea0cc11d86f60d3fde5 \
--hash=sha256:d944cea65fad0ead848b6ec2c37cc0b197194bec228f8020054742190e9312ac \
--hash=sha256:fa48268185c52bfe8771e46325a1e21d317207bcabcb72e65c6e28e9ffeb29c7 \
--hash=sha256:fc4d0d0b937e04bdf2ae6f70cd3ad51328635fe0e6214aa1fc811f3b576b3bda
safetensors==0.6.2; python_version >= '3.9' and platform_machine != 's390x' \
--hash=sha256:1d2d2b3ce1e2509c68932ca03ab8f20570920cd9754b05063d4368ee52833ecd \
--hash=sha256:43ff2aa0e6fa2dc3ea5524ac7ad93a9839256b8703761e76e2d0b2a3fa4f15d9 \
--hash=sha256:8045db2c872db8f4cbe3faa0495932d89c38c899c603f21e9b6486951a5ecb8f \
--hash=sha256:81e67e8bab9878bb568cffbc5f5e655adb38d2418351dc0859ccac158f753e19 \
--hash=sha256:89a89b505f335640f9120fac65ddeb83e40f1fd081cb8ed88b505bdccec8d0a1 \
--hash=sha256:93de35a18f46b0f5a6a1f9e26d91b442094f2df02e9fd7acf224cfec4238821a \
--hash=sha256:9c85ede8ec58f120bad982ec47746981e210492a6db876882aa021446af8ffba \
--hash=sha256:b0e4d029ab0a0e0e4fdf142b194514695b1d7d3735503ba700cf36d0fc7136ce \
--hash=sha256:c7b214870df923cbc1593c3faee16bec59ea462758699bd3fee399d00aac072c \
--hash=sha256:cab75ca7c064d3911411461151cb69380c9225798a20e712b102edda2542ddb1 \
--hash=sha256:d6675cf4b39c98dbd7d940598028f3742e0375a6b4d4277e76beb0c35f4b843b \
--hash=sha256:d83c20c12c2d2f465997c51b7ecb00e407e5f94d7dec3ea0cc11d86f60d3fde5 \
--hash=sha256:d944cea65fad0ead848b6ec2c37cc0b197194bec228f8020054742190e9312ac \
--hash=sha256:fa48268185c52bfe8771e46325a1e21d317207bcabcb72e65c6e28e9ffeb29c7 \
--hash=sha256:fc4d0d0b937e04bdf2ae6f70cd3ad51328635fe0e6214aa1fc811f3b576b3bda
🤖 Prompt for AI Agents 
In jupyter/pytorch+llmcompressor/ubi9-python-3.11/requirements.txt around lines
2874 to 2889, safetensors version 0.6.2 does not provide a prebuilt manylinux
wheel for the s390x architecture, causing pip to attempt a source build that
requires Rust. To fix this, add a platform-specific marker to exclude s390x from
installing safetensors or ensure the Rust toolchain is installed in the s390x
environment before pip installs. This gating prevents unexpected build failures
on s390x systems.

scikit-learn==1.6.1; python_version >= '3.9' \
--hash=sha256:0650e730afb87402baa88afbf31c07b84c98272622aaba002559b614600ca691 \
--hash=sha256:0c8d036eb937dbb568c6242fa598d551d88fb4399c0344d95c001980ec1c7d36 \
Expand Down Expand Up @@ -4118,9 +4118,9 @@ numba==0.61.2; python_version >= '3.10' \
--hash=sha256:cf9f9fc00d6eca0c23fc840817ce9f439b9f03c8f03d6246c0e7f0cb15b7162a \
--hash=sha256:ea0247617edcb5dd61f6106a56255baab031acc4257bddaeddb3a1003b4ca3fd \
--hash=sha256:efd3db391df53aaa5cfbee189b6c910a5b471488749fd6606c3f33fc984c2ae2
openai==1.99.3; python_version >= '3.8' \
--hash=sha256:1a0e2910e4545d828c14218f2ac3276827c94a043f5353e43b9413b38b497897 \
--hash=sha256:c786a03f6cddadb5ee42c6d749aa4f6134fe14fdd7d69a667e5e7ce7fd29a719
openai==1.99.5; python_version >= '3.8' \
--hash=sha256:4e870f9501b7c36132e2be13313ce3c4d6915a837e7a299c483aab6a6d4412e9 \
--hash=sha256:aa97ac3326cac7949c5e4ac0274c454c1d19c939760107ae0d3948fc26a924ca
opencv-python-headless==4.11.0.86; python_version >= '3.6' \
--hash=sha256:0e0a27c19dd1f40ddff94976cfe43066fbbe9dfbb2ec1907d66c19caef42a57b \
--hash=sha256:48128188ade4a7e517237c8e1e11a9cdf5c282761473383e77beb875bb1e61ca \
Expand Down
38 changes: 19 additions & 19 deletions jupyter/trustyai/ubi9-python-3.11/Pipfile.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading