Skip to content

Building NGINX Ingress Controller

Jump to bottom
aborkar-ibm edited this page Jul 17, 2019 · 33 revisions

Building NGINX Ingress Controller

The instructions provided below specify the steps to build NGINX Ingress Controller version 0.24.1 on Linux on IBM Z for following distributions:

  • RHEL (7.4, 7.5, 7.6)
  • SLES (12 SP4, 15)
  • Ubuntu (16.04, 18.04, 19.04)

Prerequisites:

  • For deploying NGINX Ingress Controller, you must have Kubernetes installed. Download Kubernetes binary from Kubernetes.

  • At the time of creation of these build instructions NGINX Ingress Controller was verified using Kubernetes version 1.13.2 and Docker 18.06.

  • Docker packages are provided for SLES, Ubuntu and RHEL (7.5 or higher) in their respective repositories. Instructions for installing Docker on RHEL(7.3) can be found here. You may use the same instructions for RHEL (7.4, 7.5, 7.6) as the binaries are expected to be compatible. More information about Docker CE can be found here.

General Notes:

  • When following the steps below please use a standard permission user unless otherwise specified.
  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Step 1: Build using script

If you want to build nginx-ingress-controller using manual steps, go to STEP 2.

Use the following commands to build nginx-ingress-controller using the build script. Please make sure you have wget and Docker installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/NGINX-ingress-controller/0.24.1/build_nginx-ingress-controller.sh

# Build nginx-ingress-controller
bash build_nginx-ingress-controller.sh   [Provide -t option for executing build with tests]

If the build completes successfully, go to STEP 5. In case of error, check logs for more details or go to STEP 2 to follow manual build steps.

Step 2: Install dependencies

export SOURCE_ROOT=/<source_root>/

  • RHEL (7.4, 7.5, 7.6)

    sudo yum install -y git make

  • SLES (12 SP4, 15)

    sudo zypper install -y git make docker

  • Ubuntu (16.04, 18.04, 19.04)

    sudo apt-get update
sudo apt-get install -y git make golang-1.10 docker.io
export PATH=/usr/lib/go-1.10/bin:$PATH

  • Go ( Only for RHEL and SLES ) -- Instructions for building Go can be found here

Step 3: Build NGINX Ingress Controller

export GOPATH=$SOURCE_ROOT
export DOCKER=docker
mkdir -p $GOPATH/src/k8s.io/
cd $GOPATH/src/k8s.io/
git clone https://github.com/kubernetes/ingress-nginx.git
cd ingress-nginx/
git checkout nginx-0.24.1

Make changes to files to build quay.io/kubernetes-ingress-controller/nginx-ingress-controller-s390x docker image. Create a patch file named patch.diff:

diff --git a/Makefile b/Makefile
index 3eb84e8..f947773 100644
--- a/Makefile
+++ b/Makefile
@@ -49,7 +49,7 @@ KUBECTL_CONTEXT = $(shell kubectl config current-context)
 
 GOBUILD_FLAGS :=
 
-ALL_ARCH = amd64 arm64
+ALL_ARCH = amd64 arm64 s390x
 
 QEMUVERSION = v3.0.0
 
@@ -65,6 +65,9 @@ BASEIMAGE?=quay.io/kubernetes-ingress-controller/nginx-$(ARCH):0.84
 ifeq ($(ARCH),arm64)
 	QEMUARCH=aarch64
 endif
+ifeq ($(ARCH),s390x)
+	QEMUARCH=s390x
+endif
 
 TEMP_DIR := $(shell mktemp -d)
 
@@ -120,7 +123,7 @@ else
 	$(SED_I) "s/CROSS_BUILD_//g" $(DOCKERFILE)
 endif
 
-	$(DOCKER) build --no-cache --pull -t $(MULTI_ARCH_IMG):$(TAG) $(TEMP_DIR)/rootfs
+	$(DOCKER) build --no-cache -t $(MULTI_ARCH_IMG):$(TAG) $(TEMP_DIR)/rootfs
 
 ifeq ($(ARCH), amd64)
 	# This is for maintaining backward compatibility
diff --git a/build/go-in-docker.sh b/build/go-in-docker.sh
index 5d3e8d5..fb8934f 100755
--- a/build/go-in-docker.sh
+++ b/build/go-in-docker.sh
@@ -40,7 +40,8 @@ if [ "$missing" = true ];then
   exit 1
 fi
 
-E2E_IMAGE=quay.io/kubernetes-ingress-controller/e2e:v04062019-3dc4253b7
+IMAGE_TAG=$(sudo docker images | grep e2e | awk '{print $2}')
+E2E_IMAGE=quay.io/kubernetes-ingress-controller/e2e:$IMAGE_TAG
 
 DOCKER_OPTS=${DOCKER_OPTS:-""}
 
diff --git a/build/test.sh b/build/test.sh
index b206a5e..4384aa4 100755
--- a/build/test.sh
+++ b/build/test.sh
@@ -23,5 +23,5 @@ if [ -z "${PKG}" ]; then
     exit 1
 fi
 
-go test -v -race -tags "cgo" \
+go test -v -tags "cgo" \
     $(go list ${PKG}/... | grep -v vendor | grep -v '/test/e2e' | grep -v images | grep -v "docs/examples")
diff --git a/deploy/mandatory.yaml b/deploy/mandatory.yaml
index 921bc48..89a29d6 100644
--- a/deploy/mandatory.yaml
+++ b/deploy/mandatory.yaml
@@ -211,7 +211,7 @@ spec:
       serviceAccountName: nginx-ingress-serviceaccount
       containers:
         - name: nginx-ingress-controller
-          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.24.1
+          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller-s390x:0.24.1
           args:
             - /nginx-ingress-controller
             - --configmap=$(POD_NAMESPACE)/nginx-configuration
diff --git a/deploy/with-rbac.yaml b/deploy/with-rbac.yaml
index 3e3661c..462d7bd 100644
--- a/deploy/with-rbac.yaml
+++ b/deploy/with-rbac.yaml
@@ -24,7 +24,7 @@ spec:
       serviceAccountName: nginx-ingress-serviceaccount
       containers:
         - name: nginx-ingress-controller
-          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.24.1
+          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller-s390x:0.24.1
           args:
             - /nginx-ingress-controller
             - --configmap=$(POD_NAMESPACE)/nginx-configuration
diff --git a/images/e2e/Dockerfile b/images/e2e/Dockerfile
index d9c5627..32653d4 100644
--- a/images/e2e/Dockerfile
+++ b/images/e2e/Dockerfile
@@ -12,7 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM quay.io/kubernetes-ingress-controller/nginx-amd64:0.84
+FROM quay.io/kubernetes-ingress-controller/nginx-s390x:0.84
 
 RUN clean-install \
   g++ \
@@ -26,8 +26,8 @@ RUN clean-install \
   pkg-config
 
 ENV GOLANG_VERSION 1.12.1
-ENV GO_ARCH        linux-amd64
-ENV GOLANG_SHA     2a3fdabf665496a0db5f41ec6af7a9b15a49fbe71a85a50ca38b1f13a103aeec
+ENV GO_ARCH        linux-s390x
+ENV GOLANG_SHA     a9b8f49be6b2083e2586c2ce8a2a86d5dbf47cca64ac6195546a81c9927f9513
 
 RUN set -eux; \
   url="https://golang.org/dl/go${GOLANG_VERSION}.${GO_ARCH}.tar.gz"; \
diff --git a/images/nginx/Makefile b/images/nginx/Makefile
index a179bc3..251d9be 100644
--- a/images/nginx/Makefile
+++ b/images/nginx/Makefile
@@ -18,7 +18,7 @@ REGISTRY ?= quay.io/kubernetes-ingress-controller
 ARCH ?= $(shell go env GOARCH)
 DOCKER ?= docker
 
-ALL_ARCH = amd64 arm64
+ALL_ARCH = amd64 arm64 s390x
 SED_I?=sed -i
 GOHOSTOS ?= $(shell go env GOHOSTOS)
 
@@ -38,7 +38,9 @@ BASEIMAGE?=quay.io/kubernetes-ingress-controller/debian-base-$(ARCH):0.1
 ifeq ($(ARCH),arm64)
 	QEMUARCH=aarch64
 endif
-
+ifeq ($(ARCH),s390x)
+	QEMUARCH=s390x
+endif
 TEMP_DIR := $(shell mktemp -d)
 
 all: all-container
diff --git a/images/nginx/rootfs/build.sh b/images/nginx/rootfs/build.sh
index 28aefc6..cdd84ce 100755
--- a/images/nginx/rootfs/build.sh
+++ b/images/nginx/rootfs/build.sh
@@ -107,6 +107,13 @@ if [[ ${ARCH} == "aarch64" ]]; then
   ln -s /usr/lib/aarch64-linux-gnu/liblua5.1.so /usr/lib/liblua.so
   ln -s /usr/lib/aarch64-linux-gnu /usr/lib/lua-platform-path
 fi
+if [[ ${ARCH} == "s390x" ]]; then
+  ln -s /usr/lib/s390x-linux-gnu/liblua5.1.so /usr/lib/liblua.so
+  ln -s /usr/lib/s390x-linux-gnu /usr/lib/lua-platform-path
+  # avoid error:
+  # git: ../nptl/pthread_mutex_lock.c:81: __pthread_mutex_lock: Assertion `mutex->__data.__owner == 0' failed.
+  git config --global pack.threads "1"
+fi
 
 mkdir -p /etc/nginx
 
@@ -234,6 +241,14 @@ export LUAJIT_LIB=/usr/local/lib
 export LUA_LIB_DIR="$LUAJIT_LIB/lua"
 
 cd "$BUILD_PATH/luajit2-$LUAJIT_VERSION"
+
+if [[ ${ARCH} == "s390x" ]]; then
+  wget https://github.com/LuaJIT/LuaJIT/compare/v2.1...linux-on-ibm-z:v2.1.diff -O s390x.patch
+  patch -p1 < s390x.patch
+  sed -i '144i #if LJ_HASJIT' src/lib_jit.c
+  sed -i '155i #endif' src/lib_jit.c
+fi
+
 make CCDEBUG=-g
 make install
 
@@ -248,6 +263,10 @@ if [[ ${ARCH} == "aarch64" ]]; then
   export PCRE_DIR=/usr/lib/aarch64-linux-gnu
 fi
 
+if [[ ${ARCH} == "s390x" ]]; then
+  export PCRE_DIR=/usr/lib/s390x-linux-gnu
+fi
+
 cd "$BUILD_PATH"
 luarocks install lrexlib-pcre 2.7.2-1 PCRE_LIBDIR=${PCRE_DIR}
diff --git a/rootfs/Dockerfile b/rootfs/Dockerfile
index d6d522f..74c2542 100644
--- a/rootfs/Dockerfile
+++ b/rootfs/Dockerfile
@@ -22,7 +22,7 @@ RUN clean-install \
   diffutils \
   libcap2-bin

-COPY --chown=www-data:www-data . /
+COPY  . /

 # Fix permission during the build to avoid issues at runtime
 # with volumes (custom templates)
 
git apply patch.diff

  • Build nginx, e2e image for s390x

    cd $GOPATH/src/k8s.io/ingress-nginx/images/nginx/
make container
cd $GOPATH/src/k8s.io/ingress-nginx/images/e2e/
make docker-build

  • Build NGINX Ingress Controller with the nginx-s390x image

    cd $GOPATH/src/k8s.io/ingress-nginx/
make build container

    After executing the above steps, these docker images should have been created:

    quay.io/kubernetes-ingress-controller/nginx-s390x
quay.io/kubernetes-ingress-controller/e2e
quay.io/kubernetes-ingress-controller/nginx-ingress-controller-s390x

    Note: Dockerfile commands used to build the NGINX Ingress Controller docker image require Docker 17.09.0-ce and above.

Step 4: Testing (Optional)

  • Run test cases

    cd $GOPATH/src/k8s.io/ingress-nginx/
make test

Step 5: Deployment

  • Deployment

    cd $GOPATH/src/k8s.io/ingress-nginx/
kubectl apply -f deploy/mandatory.yaml

Step 6: Verify installed version

POD_NAMESPACE=ingress-nginx
POD_NAME=$(kubectl get pods --all-namespaces | grep nginx-ingress-controller | awk '{print $2}')
kubectl exec -it $POD_NAME -n $POD_NAMESPACE -- /nginx-ingress-controller --version

Output should be:

-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:    0.24.1
  Build:      git-ce418168f
  Repository: https://github.com/kubernetes/ingress-nginx.git
-------------------------------------------------------------------------------

References:

NGINX Ingress Controller GitHub

The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue or contact us on IBM Z Community if you have any questions or feedback.

Clone this wiki locally