CONSOLIDATED COMMITS: Centos stream9 PyTorch support

====================================================

[SOW MS3] Centos stream9 PyTorch image support (#1090)

* changes to build Centos stream 9 images

* Added scripts for centos and centos stream images

* Added an extra line

* Add ninja installation

* Optimized code

* Fixes

* Add comment

* Optimized code

* Added AMDGPU mapping for ROCm 5.2 and invalid-url for rocm_baseurl

Co-authored-by: Jithun Nair <[email protected]>

Updated to latest conda for CentOS stream 9

[CS9] Updates to CentOS stream 9 build (#1326)

- Add missing common_utils.sh
- Update the install vision part
- Move to amdgpu rhel 9.3 builds
- Update to pick python from conda path
- Add a missing package
- Add ROCM_PATH and magma
- Updated repo radeon path

(cherry picked from commit 51ce1cc)

[rocm6.4_internal_testing] Update missing changes for CentOS9 (#1813)

To fix, https://ontrack-internal.amd.com/browse/SWDEV-505385 and
https://ontrack-internal.amd.com/browse/SWDEV-507301

(cherry picked from commit 956c145)

delete .ci/docker/common/install_db.sh

(cherry picked from commit 8a7fd64)

CONSOLIDATED COMMITS: Updates to build on Jammy and CentOS7

===========================================================

Updates to build on Jammy
- Fortran package installation moved after gcc
- Update libtinfo search code in cmake1
- Install libstdc++.so

[UB22.04] Updates to support latest scipy

Build required version of libpng for CentOS7

Updated condition for libstc++ for Jammy

Set ROCM_PATH in env for centOS docker container

Changes to support docker v23

Reversed the condition as required

temporarily ignore certificate check for Miniconda

(cherry picked from commit 9848db1)

[release/2.1] Skip certificate check for CentOS7 since certificate expired (#1399)

* Skip certificate check only for CentOS7 since certificate expired

* Naming

Remove the installation of rocm-llvm-dev package

- Causing regression - SWDEV-463083

fix install_centos() function

[rocm6.3_internal_testing] skip pytorch-nightly installstion (#1557)

This PR skips pytorch-nightly installation in docker images

Installation of pytorch-nightly is needed to prefetch mobilenet_v2 avd
v3 models for some tests.
Came from

85bd6bc
Models are downloaded on first use to the folder /root/.cache/torch/hub
But pytorch-nightly installation also overrides
.ci/docker/requirements-ci.txt settings and upgrades some of python
packages (sympy from 1.12.0 to 1.13.0) which causes several
'dynamic_shapes' tests to fail
Skip prefetching models affects these tests without any errors (but
**internet access required**):

- python test/mobile/model_test/gen_test_model.py mobilenet_v2
- python test/quantization/eager/test_numeric_suite_eager.py -k
test_mobilenet_v3

Issue ROCm/frameworks-internal#8772

Also, in case of some issues these models can be prefetched after
pytorch building and before testing

(cherry picked from commit b92b34d)

Fixes #ISSUE_NUMBER

(cherry picked from commit ec70f7e)

[rocm6.4_internal_testing] Changes to support UB 24.04 build (#1817)

Changes applied from #1816

Test PyTorch build:
http://rocm-ci.amd.com/job/mainline-framework-pytorch-ub24.04-py312-internal/5/

(cherry picked from commit 74e1e9e)
(cherry picked from commit e7cb7cc)

Update Centos 9 build

(cherry picked from commit 3d6ba22)

[rocm6.5_internal_testing] remove centos.stream dockerfile and move contents into dockerfile (#2044)

rocm6.5_internal_testing move contents of centos stream dockerfile into
dockerfile

Validation:
http://rocm-ci.amd.com/job/mainline-framework-pytorch-ci/2448/

---------

Co-authored-by: Jithun Nair <[email protected]>
(cherry picked from commit 7886773)

Author: 2 people

.ci/docker/build.sh

@@ -326,6 +326,10 @@ if [[ -n "${CI:-}" ]]; then
   progress_flag="--progress=plain"
 fi
 
+if [[ "${DOCKER_BUILDKIT}" == 0 ]]; then
+  progress_flag=""
+fi
+
 # Build image
 docker build \
        ${no_cache_flag} \

.ci/docker/centos-rocm/Dockerfile

@@ -1,8 +1,7 @@
 ARG CENTOS_VERSION
 
-FROM centos:${CENTOS_VERSION}
+FROM quay.io/centos/centos:stream${CENTOS_VERSION}
 
-ARG CENTOS_VERSION
 
 # Set AMD gpu targets to build for
 ARG PYTORCH_ROCM_ARCH
@@ -14,6 +13,9 @@ ENV PYTORCH_ROCM_ARCH ${PYTORCH_ROCM_ARCH}
 COPY ./common/install_base.sh install_base.sh
 RUN bash ./install_base.sh && rm install_base.sh
 
+#Install langpack
+RUN yum install -y glibc-langpack-en
+
 # Update CentOS git version
 RUN yum -y remove git
 RUN yum -y remove git-*
@@ -22,11 +24,12 @@ RUN yum -y install https://packages.endpointdev.com/rhel/7/os/x86_64/endpoint-re
 RUN yum install -y git
 
 # Install devtoolset
-ARG DEVTOOLSET_VERSION
-COPY ./common/install_devtoolset.sh install_devtoolset.sh
-RUN bash ./install_devtoolset.sh && rm install_devtoolset.sh
+RUN dnf install -y rpmdevtools
 ENV BASH_ENV "/etc/profile"
 
+# Install ninja
+RUN dnf --enablerepo=crb install -y ninja-build
+
 # (optional) Install non-default glibc version
 ARG GLIBC_VERSION
 COPY ./common/install_glibc.sh install_glibc.sh
@@ -69,6 +72,8 @@ RUN rm install_rocm_magma.sh
 COPY ./common/install_amdsmi.sh install_amdsmi.sh
 RUN bash ./install_amdsmi.sh
 RUN rm install_amdsmi.sh
+
+ENV ROCM_PATH /opt/rocm
 ENV PATH /opt/rocm/bin:$PATH
 ENV PATH /opt/rocm/hcc/bin:$PATH
 ENV PATH /opt/rocm/hip/bin:$PATH

.ci/docker/common/cache_vision_models.sh

@@ -2,6 +2,20 @@
 
 set -ex
 
+# Skip pytorch-nightly installation in docker images
+# Installation of pytorch-nightly is needed to prefetch mobilenet_v2 avd v3 models for some tests.
+# Came from https://github.com/ROCm/pytorch/commit/85bd6bc0105162293fa0bbfb7b661f85ec67f85a
+# Models are downloaded on first use to the folder /root/.cache/torch/hub
+# But pytorch-nightly installation also overrides .ci/docker/requirements-ci.txt settings
+# and upgrades some of python packages (sympy from 1.12.0 to 1.13.0)
+# which causes several 'dynamic_shapes' tests to fail
+# Skip prefetching models affects these tests without any errors:
+#   python test/mobile/model_test/gen_test_model.py mobilenet_v2
+#   python test/quantization/eager/test_numeric_suite_eager.py -k test_mobilenet_v3
+# Issue https://github.com/ROCm/frameworks-internal/issues/8772
+echo "Skip torch-nightly installation"
+exit 0
+
 source "$(dirname "${BASH_SOURCE[0]}")/common_utils.sh"
 
 # Cache the test models at ~/.cache/torch/hub/

.ci/docker/common/install_base.sh

@@ -77,15 +77,50 @@ install_ubuntu() {
   # see: https://github.com/pytorch/pytorch/issues/65931
   apt-get install -y libgnutls30
 
+  if [[ "$UBUNTU_VERSION" == "22.04"* ]]; then
+    apt-get install -y libopenblas-dev
+  fi
+
   # Cleanup package manager
   apt-get autoclean && apt-get clean
   rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
 }
 
+build_libpng() {
+  # install few packages
+  yum install -y zlib zlib-devel
+
+  LIBPNG_VERSION=1.6.37
+
+  mkdir -p libpng
+  pushd libpng
+
+  wget http://download.sourceforge.net/libpng/libpng-$LIBPNG_VERSION.tar.gz
+  tar -xvzf libpng-$LIBPNG_VERSION.tar.gz
+
+  pushd libpng-$LIBPNG_VERSION
+
+  ./configure
+  make
+  make install
+
+  popd
+
+  popd
+  rm -rf libpng
+}
+
 install_centos() {
   # Need EPEL for many packages we depend on.
   # See http://fedoraproject.org/wiki/EPEL
-  yum --enablerepo=extras install -y epel-release
+  # extras repo is not there for CentOS 9 and epel-release is already part of repo list
+  if [[ $OS_VERSION == 9 ]]; then
+      yum install -y epel-release
+      ALLOW_ERASE="--allowerasing"
+  else
+      yum --enablerepo=extras install -y epel-release
+      ALLOW_ERASE=""
+  fi
 
   ccache_deps="asciidoc docbook-dtds docbook-style-xsl libxslt"
   numpy_deps="gcc-gfortran"
@@ -106,24 +141,39 @@ install_centos() {
     glibc-headers \
     glog-devel \
     libstdc++-devel \
-    libsndfile-devel \
     make \
-    opencv-devel \
     sudo \
     wget \
     vim \
     unzip \
     gdb
 
+  if [[ $OS_VERSION == 9 ]]
+  then
+	  dnf --enablerepo=crb -y install libsndfile-devel
+	  yum install -y procps
+  else
+	  yum install -y \
+            opencv-devel \
+	    libsndfile-devel
+  fi
+
+  # CentOS7 doesnt have support for higher version of libpng,
+  # so it is built from source.
+  # Libpng is required for torchvision build.
+  build_libpng
+
   # Cleanup
   yum clean all
   rm -rf /var/cache/yum
   rm -rf /var/lib/yum/yumdb
   rm -rf /var/lib/yum/history
 }
 
-# Install base packages depending on the base OS
 ID=$(grep -oP '(?<=^ID=).+' /etc/os-release | tr -d '"')
+OS_VERSION=$(grep -oP '(?<=^VERSION_ID=).+' /etc/os-release | tr -d '"')
+
+# Install base packages depending on the base OS
 case "$ID" in
   ubuntu)
     install_ubuntu

.ci/docker/common/install_conda.sh

@@ -24,7 +24,10 @@ if [ -n "$ANACONDA_PYTHON_VERSION" ]; then
   source "${SCRIPT_FOLDER}/common_utils.sh"
 
   pushd /tmp
-  wget -q "${BASE_URL}/${CONDA_FILE}"
+  if [ -n $CENTOS_VERSION ] && [[ $CENTOS_VERSION == 7.* ]]; then
+    NO_CHECK_CERTIFICATE_FLAG="--no-check-certificate"
+  fi
+  wget -q "${BASE_URL}/${CONDA_FILE}" ${NO_CHECK_CERTIFICATE_FLAG}
   # NB: Manually invoke bash per https://github.com/conda/conda/issues/10431
   as_jenkins bash "${CONDA_FILE}" -b -f -p "/opt/conda"
   popd
@@ -40,8 +43,13 @@ if [ -n "$ANACONDA_PYTHON_VERSION" ]; then
 
   # Prevent conda from updating to 4.14.0, which causes docker build failures
   # See https://hud.pytorch.org/pytorch/pytorch/commit/754d7f05b6841e555cea5a4b2c505dd9e0baec1d
-  # Uncomment the below when resolved to track the latest conda update
-  # as_jenkins conda update -y -n base conda
+  # Uncomment the below when resolved to track the latest conda update,
+  # but this is required for CentOS stream 9 builds
+  ID=$(grep -oP '(?<=^ID=).+' /etc/os-release | tr -d '"')
+  OS_VERSION=$(grep -oP '(?<=^VERSION_ID=).+' /etc/os-release | tr -d '"')
+  if [[ $ID == centos && $OS_VERSION == 9 ]]; then
+    as_jenkins conda update -y -n base conda
+  fi
 
   if [[ $(uname -m) == "aarch64" ]]; then
     export SYSROOT_DEP="sysroot_linux-aarch64=2.17"
@@ -86,6 +94,15 @@ if [ -n "$ANACONDA_PYTHON_VERSION" ]; then
     conda_install_through_forge libstdcxx-ng=14
   fi
 
+  # Install required libstdc++.so.6 version
+  if [ "$ANACONDA_PYTHON_VERSION" = "3.10" ] || [ "$ANACONDA_PYTHON_VERSION" = "3.9" ] ; then
+    conda_install_through_forge libstdcxx-ng=12
+  fi
+
+  if [ "$ANACONDA_PYTHON_VERSION" = "3.12" ] || [ "$UBUNTU_VERSION" == "24.04"* ] ; then
+    conda_install_through_forge libstdcxx-ng=14
+  fi
+
   # Install some other packages, including those needed for Python test reporting
   pip_install -r /opt/conda/requirements-ci.txt
 

.ci/docker/common/install_rocm.sh

@@ -60,10 +60,6 @@ EOF
                    roctracer-dev \
                    amd-smi-lib
 
-    if [[ $(ver $ROCM_VERSION) -ge $(ver 6.1) ]]; then
-        DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated rocm-llvm-dev
-    fi
-
     # precompiled miopen kernels added in ROCm 3.5, renamed in ROCm 5.5
     # search for all unversioned packages
     # if search fails it will abort this script; use true to avoid case where search fails
@@ -124,44 +120,64 @@ install_centos() {
   yum update -y
   yum install -y kmod
   yum install -y wget
-  yum install -y openblas-devel
+  
+  if [[ $OS_VERSION == 9 ]]; then 
+      dnf install -y openblas-serial
+      dnf install -y dkms kernel-headers kernel-devel
+  else
+      yum install -y openblas-devel
+      yum install -y dkms kernel-headers-`uname -r` kernel-devel-`uname -r`
+  fi
 
   yum install -y epel-release
-  yum install -y dkms kernel-headers-`uname -r` kernel-devel-`uname -r`
 
-  # Add amdgpu repository
-  local amdgpu_baseurl
+  if [[ $(ver $ROCM_VERSION) -ge $(ver 4.5) ]]; then
+      # Add amdgpu repository
+      local amdgpu_baseurl
+      if [[ $OS_VERSION == 9 ]]; then
+          amdgpu_baseurl="https://repo.radeon.com/amdgpu/${AMDGPU_VERSIONS[$ROCM_VERSION]}/rhel/9.1/main/x86_64"
+      else
+        if [[ $(ver $ROCM_VERSION) -ge $(ver 5.3) ]]; then
+          amdgpu_baseurl="https://repo.radeon.com/amdgpu/${ROCM_VERSION}/rhel/7.9/main/x86_64"
+        else
+          amdgpu_baseurl="https://repo.radeon.com/amdgpu/${AMDGPU_VERSIONS[$ROCM_VERSION]}/rhel/7.9/main/x86_64"
+        fi
+      fi
+      echo "[AMDGPU]" > /etc/yum.repos.d/amdgpu.repo
+      echo "name=AMDGPU" >> /etc/yum.repos.d/amdgpu.repo
+      echo "baseurl=${amdgpu_baseurl}" >> /etc/yum.repos.d/amdgpu.repo
+      echo "enabled=1" >> /etc/yum.repos.d/amdgpu.repo
+      echo "gpgcheck=1" >> /etc/yum.repos.d/amdgpu.repo
+      echo "gpgkey=http://repo.radeon.com/rocm/rocm.gpg.key" >> /etc/yum.repos.d/amdgpu.repo
+  fi
+
   if [[ $OS_VERSION == 9 ]]; then
-      amdgpu_baseurl="https://repo.radeon.com/amdgpu/${ROCM_VERSION}/rhel/9.0/main/x86_64"
+      local rocm_baseurl="invalid-url"
   else
-      amdgpu_baseurl="https://repo.radeon.com/amdgpu/${ROCM_VERSION}/rhel/7.9/main/x86_64"
+      local rocm_baseurl="http://repo.radeon.com/rocm/yum/${ROCM_VERSION}/main"
   fi
-  echo "[AMDGPU]" > /etc/yum.repos.d/amdgpu.repo
-  echo "name=AMDGPU" >> /etc/yum.repos.d/amdgpu.repo
-  echo "baseurl=${amdgpu_baseurl}" >> /etc/yum.repos.d/amdgpu.repo
-  echo "enabled=1" >> /etc/yum.repos.d/amdgpu.repo
-  echo "gpgcheck=1" >> /etc/yum.repos.d/amdgpu.repo
-  echo "gpgkey=http://repo.radeon.com/rocm/rocm.gpg.key" >> /etc/yum.repos.d/amdgpu.repo
-
-  local rocm_baseurl="http://repo.radeon.com/rocm/yum/${ROCM_VERSION}"
   echo "[ROCm]" > /etc/yum.repos.d/rocm.repo
   echo "name=ROCm" >> /etc/yum.repos.d/rocm.repo
   echo "baseurl=${rocm_baseurl}" >> /etc/yum.repos.d/rocm.repo
   echo "enabled=1" >> /etc/yum.repos.d/rocm.repo
   echo "gpgcheck=1" >> /etc/yum.repos.d/rocm.repo
   echo "gpgkey=http://repo.radeon.com/rocm/rocm.gpg.key" >> /etc/yum.repos.d/rocm.repo
 
-  yum update -y
-
-  yum install -y \
+  if [[ $OS_VERSION == 9 ]]; then
+      yum update -y --nogpgcheck
+      dnf --enablerepo=crb install -y perl-File-BaseDir python3-wheel
+      yum install -y --nogpgcheck rocm-ml-sdk rocm-developer-tools
+  else
+      yum update -y
+      yum install -y \
                    rocm-dev \
                    rocm-utils \
                    rocm-libs \
                    rccl \
                    rocprofiler-dev \
                    roctracer-dev \
                    amd-smi-lib
-
+  fi
   # precompiled miopen kernels; search for all unversioned packages
   # if search fails it will abort this script; use true to avoid case where search fails
   MIOPENHIPGFX=$(yum -q search miopen-hip-gfx | grep miopen-hip-gfx | awk '{print $1}'| grep -F kdb. || true)
@@ -186,6 +202,8 @@ install_centos() {
   rm -rf /var/lib/yum/history
 }
 
+OS_VERSION=$(grep -oP '(?<=^VERSION_ID=).+' /etc/os-release | tr -d '"')
+
 # Install Python packages depending on the base OS
 ID=$(grep -oP '(?<=^ID=).+' /etc/os-release | tr -d '"')
 case "$ID" in

.ci/docker/common/install_vision.sh

@@ -15,10 +15,14 @@ install_ubuntu() {
 install_centos() {
   # Need EPEL for many packages we depend on.
   # See http://fedoraproject.org/wiki/EPEL
-  yum --enablerepo=extras install -y epel-release
-
-  yum install -y \
-      opencv-devel
+  if [[ $OS_VERSION == 9 ]]; then
+      yum install -y epel-release
+  else
+      yum --enablerepo=extras install -y epel-release
+      yum install -y \
+          opencv-devel \
+          ffmpeg-devel
+  fi
 
   # Cleanup
   yum clean all
@@ -27,6 +31,8 @@ install_centos() {
   rm -rf /var/lib/yum/history
 }
 
+OS_VERSION=$(grep -oP '(?<=^VERSION_ID=).+' /etc/os-release | tr -d '"')
+
 # Install base packages depending on the base OS
 ID=$(grep -oP '(?<=^ID=).+' /etc/os-release | tr -d '"')
 case "$ID" in

.ci/docker/requirements-ci.txt

@@ -117,6 +117,8 @@ ninja==1.11.1.4
 #Pinned versions: 1.11.1.4
 #test that import: run_test.py, test_cpp_extensions_aot.py,test_determination.py
 
+numba==0.49.0 ; python_version < "3.9"
+numba==0.55.2 ; python_version == "3.9"
 numba==0.55.2 ; python_version == "3.10" and platform_machine != "s390x"
 numba==0.60.0 ; python_version == "3.12" and platform_machine != "s390x"
 #Description: Just-In-Time Compiler for Numerical Functions