Building GlusterFS

Below versions of GlusterFS are available in respective distributions at the time of creation of these build instructions:

Ubuntu 16.04 - 3.7.6
Ubuntu 18.04 - 3.13.2
Ubuntu 19.04 - 5.3

The instructions provided below specify the steps to build GlusterFS 6.1 on Linux on IBM Z for following distributions:

RHEL (7.4, 7.5, 7.6)
SLES (12 SP4, 15)
Ubuntu (16.04, 18.04, 19.04)

General Note:

When following the steps below please use a super user unless otherwise specified
A directory /<source root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it

Building and Installing GlusterFS

Step 1: Build using script

If you want to build GlusterFS using manual steps, go to Step 2.

Use the following commands to build GlusterFS using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/GlusterFS/build_glusterfs.sh
# Build GlusterFS
bash build_glusterfs.sh   [Provide -t option for executing build with tests]

If the build completes successfully, go to STEP 5. In case of error, check logs for more details or go to STEP 2 to follow manual build steps.

Step 2: Install the dependencies

export SOURCE_ROOT=/<source_root>/

RHEL (7.4, 7.5, 7.6)

yum install -y autoconf automake bison bzip2 flex fuse-devel gcc git glib2-devel libacl-devel libaio-devel libibverbs-devel librdmacm-devel libtool libxml2-devel libuuid-devel lvm2 make openssl-devel pkgconfig python readline-devel wget zlib-devel

Building urcu

cd $SOURCE_ROOT
wget https://lttng.org/files/urcu/userspace-rcu-0.10.2.tar.bz2
tar xvjf userspace-rcu-0.10.2.tar.bz2
cd userspace-rcu-0.10.2
./configure --prefix=/usr --libdir=/usr/lib64
make
make install

SLES (12 SP4, 15)

zypper install -y autoconf automake bison cmake flex fuse-devel gcc git glib2-devel libacl-devel libaio-devel librdmacm1 libopenssl-devel libtool liburcu-devel libuuid-devel libxml2-devel lvm2 make pkg-config python3 rdma-core-devel readline-devel zlib-devel

Ubuntu (16.04, 18.04, 19.04)

apt-get update
apt-get install -y autoconf automake bison flex gcc git libacl1-dev libaio-dev libfuse-dev libglib2.0-dev libibverbs-dev librdmacm-dev libreadline-dev libssl-dev libtool liburcu-dev libxml2-dev lvm2 make openssl pkg-config python3 uuid-dev zlib1g-dev

Step 3: Download, Configure and Build GlusterFS

Download and configure source

cd SOURCE_ROOT
git clone -b v6.1 https://github.com/gluster/glusterfs
cd glusterfs
./autogen.sh
./configure --enable-gnfs                       # For RHEL, SLES 15 and Ubuntu
./configure --enable-gnfs --disable-events      # For SLES 12 SP4

Create a patch file patch_io-threads.diff with the following contents: (For SLES and Ubuntu only)

diff --git a/xlators/performance/io-threads/src/io-threads.h b/xlators/performance/io-threads/src/io-threads.h
index f54d2f4..1d9c22e 100644
--- a/xlators/performance/io-threads/src/io-threads.h
+++ b/xlators/performance/io-threads/src/io-threads.h
@@ -34,7 +34,7 @@ struct iot_conf;
 #define IOT_DEFAULT_THREADS 16
 #define IOT_MAX_THREADS 64

-#define IOT_THREAD_STACK_SIZE ((size_t)(256 * 1024))
+#define IOT_THREAD_STACK_SIZE ((size_t)(512 * 1024))

 typedef struct {
     struct list_head clients;

Apply the patch using the command below:

git apply patch_io-threads.diff

Build and install
```
make
make install
ldconfig
```

Step 4: Run the test cases (optional)

4.1) Install the testing dependencies

RHEL (7.4, 7.5, 7.6)

yum install -y acl attr bc bind-utils boost-devel docbook-style-xsl expat-devel gcc-c++ gdb net-tools nfs-utils psmisc pyxattr vim xfsprogs yajl

SLES 12 SP4

zypper install -y acl attr bc bind-utils boost-devel gcc-c++ gdb libexpat-devel libxml2-tools net-tools nfs-utils psmisc vim xfsprogs

SLES 15

zypper install -y acl attr bc bind-utils gdb libxml2-tools net-tools-deprecated nfs-utils psmisc thin-provisioning-tools vim xfsprogs

Ubuntu (16.04, 18.04, 19.04)

apt-get install -y acl attr bc dbench dnsutils libxml2-utils net-tools nfs-common psmisc python3-pyxattr thin-provisioning-tools vim xfsprogs yajl-tools

Building DBENCH (For RHEL and SLES only)

cd $SOURCE_ROOT
git clone https://github.com/sahlberg/dbench
cd dbench
./autogen.sh
./configure
make
make install

Building thin-provisioning-tools (For RHEL and SLES SP4 only)

cd $SOURCE_ROOT
git clone https://github.com/jthornber/thin-provisioning-tools
cd thin-provisioning-tools
git checkout v0.7.6
autoreconf
./configure
make
make install

Building YAJL (For SLES only)

cd $SOURCE_ROOT
git clone https://github.com/lloyd/yajl
cd yajl
git checkout 2.1.0
./configure
make install

4.2) Patch and run tests

Create a patch file patch_tests.diff with the following contents:

diff --git a/tests/basic/distribute/throttle-rebal.t b/tests/basic/distribute/throttle-rebal.t
index f4823cf..1cc3cac 100644
--- a/tests/basic/distribute/throttle-rebal.t
+++ b/tests/basic/distribute/throttle-rebal.t
@@ -17,7 +17,7 @@ function set_throttle {
 }

 #Determine number of cores
-cores=$(cat /proc/cpuinfo | grep processor | wc -l)
+cores=$(cat /proc/cpuinfo | grep -w processor | wc -l)
 if [ "$cores" == "" ]; then
         echo "Could not get number of cores available"
 fi
diff --git a/tests/bugs/distribute/bug-1161311.t b/tests/bugs/distribute/bug-1161311.t
index c52c69b..45b4c7b 100755
--- a/tests/bugs/distribute/bug-1161311.t
+++ b/tests/bugs/distribute/bug-1161311.t
@@ -89,6 +89,7 @@ done
 ## FILE1 and FILE2 hashes are, 678b1c4a e22c1ada, so they fall
 ## into separate bricks when brick count is 3
 TEST mv $M0/dir1/FILE2 $M0/dir1/FILE1
+brick_loc=$(get_backend_paths $M0/dir1/FILE1)

 # unmount and remount the volume
 EXPECT_WITHIN $UMOUNT_TIMEOUT "Y" force_umount $M0
@@ -99,7 +100,7 @@ TEST $CLI volume rebalance $V0 start force

 # Wait for FILE to get the sticky bit on, so that file is under
 # active rebalance, before creating the links
-TEST checksticky $B0/${V0}3/dir1/FILE1
+TEST checksticky $brick_loc

 # Create the links
 ## FILE3 FILE5 FILE7 have hashes, c8c91469 566d26ce 22ce7eba
@@ -120,7 +121,7 @@ cd /

 # Ideally for this test to have done its job, the file should still be
 # under migration, so check the sticky bit again
-TEST checksticky $B0/${V0}3/dir1/FILE1
+TEST checksticky $brick_loc

 # Wait for rebalance to complete
 EXPECT_WITHIN $REBALANCE_TIMEOUT "completed" rebalance_status_field $V0
diff --git a/tests/bugs/distribute/bug-1193636.t b/tests/bugs/distribute/bug-1193636.t
index ccde02e..e97d705 100644
--- a/tests/bugs/distribute/bug-1193636.t
+++ b/tests/bugs/distribute/bug-1193636.t
@@ -40,12 +40,13 @@ dd if=/dev/zero of=$M0/dir1/FILE2 bs=64k count=10240
 # Rename the file to create a linkto, for rebalance to
 # act on the file
 TEST mv $M0/dir1/FILE2 $M0/dir1/FILE1
+brick_loc=$(get_backend_paths $M0/dir1/FILE1)

 build_tester $(dirname $0)/bug-1193636.c

 TEST $CLI volume rebalance $V0 start force

-TEST checksticky $B0/${V0}3/dir1/FILE1
+TEST checksticky $brick_loc

 TEST setfattr -n "user.test1" -v "test1" $M0/dir1/FILE1
 TEST setfattr -n "user.test2" -v "test1" $M0/dir1/FILE1
diff --git a/tests/bugs/nfs/bug-847622.t b/tests/bugs/nfs/bug-847622.t
index 5ccee72..20d5ec5 100755
--- a/tests/bugs/nfs/bug-847622.t
+++ b/tests/bugs/nfs/bug-847622.t
@@ -30,7 +30,7 @@ cd $N0

 # simple getfacl setfacl commands
 TEST touch testfile
-TEST setfacl -m u:14:r testfile
+TEST setfacl -m u:14:r $B0/brick0/testfile
 TEST getfacl testfile

 cd

Apply the patch using the command below:

git apply patch_tests.diff

Create a patch file patch_run-tests.diff with the following contents:
(Only for Ubuntu 16.04 and SLES 12 SP4, reference to the issue.)

diff --git a/run-tests.sh b/run-tests.sh
index 5683b21..f2e8950 100755
--- a/run-tests.sh
+++ b/run-tests.sh
@@ -128,10 +128,10 @@ function check_dependencies()
     fi

     # Check for netstat
-    env netstat --version > /dev/null 2>&1
-    if [ $? -ne 0 ]; then
-        MISSING="$MISSING netstat"
-    fi
+    # env netstat --version > /dev/null 2>&1
+    # if [ $? -ne 0 ]; then
+    #     MISSING="$MISSING netstat"
+    # fi

     # Check for the Perl Test Harness
     env prove --version > /dev/null 2>&1
@@ -165,10 +165,10 @@ function check_dependencies()
     fi

     # Check for netstat
-    env netstat --version > /dev/null 2>&1
-    if [ $? -ne 0 ]; then
-        MISSING="$MISSING netstat"
-    fi
+    # env netstat --version > /dev/null 2>&1
+    # if [ $? -ne 0 ]; then
+    #    MISSING="$MISSING netstat"
+    # fi

     # check for psutil python package
     test `uname -s` == "Darwin" || test `uname -s` == "FreeBSD" && {

Apply the patch using the command below:

git apply patch_run-tests.diff

Run the test cases

cd $SOURCE_ROOT/glusterfs
./run-tests.sh

Note:

If any of the tests fail with error volume create: patchy: failed: Host is not in 'Peer in Cluster' state, make changes to file /etc/hosts by copying the host details entry that is present in the beginning to the section IPv6 capable hosts and replace 0.0.0.0 with the IP address of the host. It should be similar to : 0.0.0.0 .xyz.com hostname
If you want to use the kernel NFS server or client on a Gluster storage server, make sure to disable the Gluster NFS server:
```
gluster volume set <VOLUME> nfs.disable true
```
For geo-replication test case : Password-less SSH login has to be set up. Refer to the Setting Up the Environment for Geo-replication in the documentation.
The patch for io-threads.h will allow ./tests/features/weighted-rebalance.t test case to pass.
The is_nfs_export_available function from nfs.rc used in some tests may fail intermittently. You may try to enable rpcbind as that may fix some of the test cases. The community is aware this behaviour, refer here for more info.
The test failures below are observed on s390x due to the little endian/big endian specific hash values calculations. Refer to issue here for further info.
./tests/basic/changelog/changelog-rename.t
./tests/basic/namespace.t
./tests/bugs/glusterfs/bug-902610.t
./tests/bugs/posix/bug-1619720.t
./tests/bugs/replicate/bug-1655854-support-dist-to-rep3-arb-conversion.t
./tests/features/fuse-lru-limit.t
For ./tests/bitrot/bug-1207627-bitrot-scrub-status.t test case : This test failure is under investigation, refer to this issue for details.
For ./tests/features/ssl-ciphers.t test case : The test case is failing for both SLES 15 and Ubuntu 18.04. Changes have been made in master branch here.
For /tests/bugs/glusterfs/bug-866459.t test case : . This test is under investigation, refer to this issue for details.
Additional information on the GlusterFS test framework can be found here.

Step 5: Run the GlusterFS Daemon (Optional)

glusterd

Step 6: Testing the GlusterFS volume (Optional)

Refer Documentation for instructions.

Reference

https://www.gluster.org/
https://github.com/gluster/glusterfs
http://docs.gluster.org/en/latest/Developer-guide/Building-GlusterFS/

The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue or contact us on IBM Z Community if you have any questions or feedback.

Building GlusterFS

Building GlusterFS

General Note:

Building and Installing GlusterFS

Step 1: Build using script

Step 2: Install the dependencies

Step 3: Download, Configure and Build GlusterFS

Step 4: Run the test cases (optional)

4.1) Install the testing dependencies

4.2) Patch and run tests

Step 5: Run the GlusterFS Daemon (Optional)

Step 6: Testing the GlusterFS volume (Optional)

Reference

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!