Updated Dockerfile.cpu for Datascience jupyter to support on power (ppc64le) #2406
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -53,6 +53,52 @@ RUN curl -L https://mirror.openshift.com/pub/openshift-v4/$(uname -m)/clients/oc | |
| rm -f /tmp/openshift-client-linux.tar.gz | ||
| # Install the oc client end | ||
|
|
||
| ####################################################### | ||
| # common-builder (for Power-only) | ||
| ####################################################### | ||
| FROM cpu-base AS common-builder | ||
| USER root | ||
| RUN if [ "$(uname -m)" = "ppc64le" ]; then \ | ||
| dnf install -y gcc-toolset-13 cmake ninja-build git wget unzip && \ | ||
| dnf clean all; \ | ||
| else \ | ||
| echo "Skipping common-builder package install on non-Power"; \ | ||
| fi | ||
|
|
||
| ####################################################### | ||
| # onnx-builder (Power-only) | ||
| ####################################################### | ||
| FROM common-builder AS onnx-builder | ||
| ARG ONNX_VERSION=v1.19.0 | ||
| WORKDIR /root | ||
| RUN if [ "$(uname -m)" = "ppc64le" ]; then \ | ||
| source /opt/rh/gcc-toolset-13/enable && \ | ||
| git clone --recursive https://github.com/onnx/onnx.git && \ | ||
| cd onnx && git checkout ${ONNX_VERSION} && \ | ||
| git submodule update --init --recursive && \ | ||
| pip install -r requirements.txt && \ | ||
| export CMAKE_ARGS="-DPython3_EXECUTABLE=$(which python3.12)" && \ | ||
| pip wheel . -w /root/onnx_wheel; \ | ||
| else \ | ||
| echo "Skipping ONNX build on non-Power"; \ | ||
| mkdir -p /root/onnx_wheel; \ | ||
| fi | ||
|
|
||
| ####################################################### | ||
| # openblas-builder (Power-only) | ||
| ####################################################### | ||
| FROM common-builder AS openblas-builder | ||
| ARG OPENBLAS_VERSION=0.3.30 | ||
| WORKDIR /root | ||
| RUN if [ "$(uname -m)" = "ppc64le" ]; then \ | ||
| wget https://github.com/OpenMathLib/OpenBLAS/releases/download/v${OPENBLAS_VERSION}/OpenBLAS-${OPENBLAS_VERSION}.zip && \ | ||
| unzip OpenBLAS-${OPENBLAS_VERSION}.zip && \ | ||
| cd OpenBLAS-${OPENBLAS_VERSION} && \ | ||
| make -j$(nproc) TARGET=POWER9 BINARY=64 USE_OPENMP=1 USE_THREAD=1 NUM_THREADS=120 DYNAMIC_ARCH=1 INTERFACE64=0; \ | ||
| else \ | ||
| mkdir -p OpenBLAS-${OPENBLAS_VERSION}; \ | ||
| echo "Skipping OpenBLAS build on non-Power"; \ | ||
| fi | ||
| #################### | ||
| # jupyter-minimal # | ||
| #################### | ||
|
|
@@ -85,6 +131,7 @@ ENTRYPOINT ["start-notebook.sh"] | |
| FROM jupyter-minimal AS jupyter-datascience | ||
|
|
||
| ARG DATASCIENCE_SOURCE_CODE=jupyter/datascience/ubi9-python-3.12 | ||
| ARG OPENBLAS_VERSION=0.3.30 | ||
|
|
||
| LABEL name="odh-notebook-jupyter-datascience-ubi9-python-3.12" \ | ||
| summary="Jupyter data science notebook image for ODH notebooks" \ | ||
|
|
@@ -102,15 +149,32 @@ WORKDIR /opt/app-root/bin | |
| USER root | ||
|
|
||
| # Install useful OS packages | ||
| RUN dnf install -y jq unixODBC unixODBC-devel postgresql git-lfs libsndfile libxcrypt-compat && \ | ||
| dnf clean all && rm -rf /var/cache/yum | ||
|
|
||
| # Copy dynamically-linked mongocli built in earlier build stage | ||
| COPY --from=mongocli-builder /tmp/mongocli /opt/app-root/bin/ | ||
|
|
||
|
|
||
| ENV PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/ | ||
|
|
||
| # Copy OpenBLAS,ONNX wheels for Power | ||
| COPY --from=openblas-builder /root/OpenBLAS-${OPENBLAS_VERSION} /openblas | ||
| COPY --from=onnx-builder /root/onnx_wheel/ /onnxwheels/ | ||
|
|
||
| # Power-specific ONNX/OpenBLAS installation | ||
| RUN if [ "$(uname -m)" = "ppc64le" ]; then \ | ||
| pip install /onnxwheels/*.whl && \ | ||
| PREFIX=/usr/local make -C /openblas install && \ | ||
| rm -rf /onnxwheels /openblas; \ | ||
|
Comment on lines
+162
to
+169
There was a problem hiding this comment. 💡 Verification agent 🧩 Analysis chainConfirm ONNX presence in pylock.toml to avoid double-install If 🏁 Script executed: #!/bin/bash
# Confirm ONNX markers in pylock.toml for ppc64le
rg -nP '(?i)^\s*name\s*=\s*"onnx"|platform_machine.*ppc64le' jupyter/datascience/ubi9-python-3.12/pylock.toml -C3Length of output: 22231 Exclude onnx from pylock.toml for ppc64le to avoid double-install pylock.toml contains onnx (1.19.0) with no ppc64le exclusion while Dockerfile.cpu installs /onnxwheels/*.whl on ppc64le — this will produce duplicate/conflicting installs. 🤖 Prompt for AI AgentsThere was a problem hiding this comment. this is difficult; I want to compile some wheels in a separate builder stage of a multistage docker build; is there a way to give uv the result and let it install that (and error out if my builder stage built wrong version, for example, different from what's in pylock.toml?) There was a problem hiding this comment. . Given your multi-architecture build requirements, I'd recommend Option 1 with 🐰 curious ears perked up, learning about advanced build orchestration 🧠 Learnings usedThere was a problem hiding this comment. @coderabbitai previous answer is truncated, please analyze this again and print all your advice and options There was a problem hiding this comment. @jiridanek Here's a comprehensive analysis of options for using pre-built wheels from multi-stage Docker builds with uv validation: Option 1:
|
||
| else \ | ||
| echo "Skipping ONNX/OpenBLAS install on non-Power"; \ | ||
| fi | ||
|
Comment on lines
+165
to
+172
There was a problem hiding this comment. 🛠️ Refactor suggestion Fix: OpenBLAS “make install” runs in final image without make installed The final stage calls Apply this diff in the openblas-builder stage and final stage: @@
FROM common-builder AS openblas-builder
-ARG OPENBLAS_VERSION=0.3.30
+ARG OPENBLAS_VERSION=0.3.30
WORKDIR /root
RUN if [ "$(uname -m)" = "ppc64le" ]; then \
wget https://github.com/OpenMathLib/OpenBLAS/releases/download/v${OPENBLAS_VERSION}/OpenBLAS-${OPENBLAS_VERSION}.zip && \
unzip OpenBLAS-${OPENBLAS_VERSION}.zip && \
cd OpenBLAS-${OPENBLAS_VERSION} && \
- make -j$(nproc) TARGET=POWER9 BINARY=64 USE_OPENMP=1 USE_THREAD=1 NUM_THREADS=120 DYNAMIC_ARCH=1 INTERFACE64=0; \
+ make -j$(nproc) TARGET=POWER9 BINARY=64 USE_OPENMP=1 USE_THREAD=1 NUM_THREADS=120 DYNAMIC_ARCH=1 INTERFACE64=0 && \
+ make install PREFIX=/usr/local DESTDIR=/root/openblas-stage; \
else \
- mkdir -p OpenBLAS-${OPENBLAS_VERSION}; \
+ mkdir -p openblas-stage; \
echo "Skipping OpenBLAS build on non-Power"; \
fi
@@
-# Copy OpenBLAS,ONNX wheels for Power
-COPY --from=openblas-builder /root/OpenBLAS-${OPENBLAS_VERSION} /openblas
+# Copy OpenBLAS,ONNX wheels for Power
+COPY --from=openblas-builder /root/openblas-stage/usr/local/ /usr/local/
COPY --from=onnx-builder /root/onnx_wheel/ /onnxwheels/
@@
-# Power-specific ONNX/OpenBLAS installation
-RUN if [ "$(uname -m)" = "ppc64le" ]; then \
- pip install /onnxwheels/*.whl && \
- PREFIX=/usr/local make -C /openblas install && \
- rm -rf /onnxwheels /openblas; \
- else \
- echo "Skipping ONNX/OpenBLAS install on non-Power"; \
- fi
+# Power-specific ONNX installation
+RUN if [ "$(uname -m)" = "ppc64le" ]; then \
+ pip install /onnxwheels/*.whl && \
+ rm -rf /onnxwheels; \
+ else \
+ echo "Skipping ONNX install on non-Power"; \
+ fi
|
||
|
|
||
| # Other apps and tools installed as default user | ||
| USER 1001 | ||
|
|
||
| # Install Python packages and Jupyterlab extensions from pylock.toml | ||
| COPY ${DATASCIENCE_SOURCE_CODE}/pylock.toml ./ | ||
| # Copy Elyra setup to utils so that it's sourced at startup | ||
| COPY ${DATASCIENCE_SOURCE_CODE}/setup-elyra.sh ${DATASCIENCE_SOURCE_CODE}/utils ./utils/ | ||
|
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🛠️ Refactor suggestion
Prefer TARGETARCH over uname -m for build-time conditionals
Using
${TARGETARCH}is more robust in cross-builds and aligns with our multi-arch patterns. Replace$(uname -m)checks with$TARGETARCHin all conditional RUNs for Power-only steps.Also applies to: 71-86, 90-101, 165-172
🤖 Prompt for AI Agents