Skip to content

Commit c8ff00a

Browse files
committed
Add hdf5 from EPEL to tensorflow images for arm64
When installing python wheels that contain native bits, the Pipfile.lock will only contain artifact hashes for the architecture that `pip lock` was run against, along with the source archive hash. So when installing on a different architecture, pip will attempt to compile from the source archive, and therefore will need the appropriate development files for the native dependencies that are used by the programs that it's compiling. In this case, the h5py python package is needed in the tensorflow images, and to compile the native shared object files that it contains from source, the libhdf5.so file from hdf5-devel is needed. The compiled object files will be dynamically linked to the .so files from hdf5 (so technically the hdf5-devel is only needed at compile time, then hdf5 at runtime, but since the compilation is only done on _some_ architectures, there isn't a dedicated build stage for these python packages, so to try to make minimal changes, the -devel package is left in place. On the architecture that the Pipfile.lock was generated on (x86_64), the native bits are downloaded pre-compiles as before. This makes things a little weird, as on x86_64 we'll have .so files that are precompiled and link to other .so files downloaded from PyPI, whereas on aarch64 we'll have .so files that were compiled as part of the build and linked to other .so files from hdf5 and other RPMs from the system.
1 parent b81a96b commit c8ff00a

File tree

2 files changed

+10
-0
lines changed

2 files changed

+10
-0
lines changed

jupyter/tensorflow/ubi9-python-3.12/Dockerfile.cuda

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -189,6 +189,11 @@ RUN ACCEPT_EULA=Y dnf install -y mssql-tools18 unixODBC-devel && dnf clean all &
189189

190190
ENV PATH="$PATH:/opt/mssql-tools18/bin"
191191

192+
# hdf5 is needed for h5py
193+
RUN dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm && \
194+
dnf install -y hdf5-devel && \
195+
dnf clean all
196+
192197
# Other apps and tools installed as default user
193198
USER 1001
194199

runtimes/tensorflow/ubi9-python-3.12/Dockerfile.cuda

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,11 @@ RUN yum install -y \
121121
# Set this flag so that libraries can find the location of CUDA
122122
ENV XLA_FLAGS=--xla_gpu_cuda_data_dir=/usr/local/cuda
123123

124+
# hdf5 is needed for h5py
125+
RUN dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm && \
126+
dnf install -y hdf5-devel && \
127+
dnf clean all
128+
124129
# Restore notebook user workspace
125130
USER 1001
126131
WORKDIR /opt/app-root/src

0 commit comments

Comments
 (0)