Skip to content

Building ScyllaDB

Jump to bottom
aborkar-ibm edited this page Jun 30, 2020 · 39 revisions

Building ScyllaDB

The instructions provided below specify the steps to build ScyllaDB version 3.3.1 on Linux on IBM Z for the following distributions:

  • RHEL 7.6, 7.7, 7.8
  • Ubuntu 16.04, 18.04

Ensure that the kernel you are using is the latest available.

General Notes:

  • ScyllaDB and its dependencies are large projects and take a significant amount of time to build. The build can be sped up on large systems by increasing the -j parameter passed to make and/or ninja. The build was tested on a machine with 8 vCPUs and 32GB RAM.
  • Building ScyllaDB takes a large amount of disk space. Ensure you have at least 75GB available on the filesystem you are using. Significantly more space will be necessary if you wish to add debug information to the test binaries.
  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.
  • An installation directory /<prefix>/ is referred to in these instructions, by default this is /usr/local.
  • A target architecture /<target>/ is referred to in these instructions - this is the value passed to the GCC compiler.

1. Build using script

If you want to build ScyllaDB manually, go to STEP 9.

Use the following commands to build ScyllaDB using the build script. Please make sure you have wget installed.

wget -q https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/ScyllaDB/3.3.1/build_scylladb.sh

bash build_scylladb.sh [-t] [-y] -z <target>

Provide -t option for executing build with tests.
Provide -y to install required packages without confirmation. Provide -z option to select target architecture for build e.g. z13, z14

If the build completes successfully, go to STEP 6. In case of error, check logs for more details or go to STEP 2 to follow manual build steps.

2. Install Dependencies

export SOURCE_ROOT=/<source_root>/
export TARGET=native                      # z13, z14, native
export PREFIX=/usr/local                  # /<prefix>/

2.1 Prepare for installation

  • RHEL 7.8

    Downgrade broken glibc version if installed.

    sudo yum list installed glibc-2.17-307.el7.1.s390
if [[ $? ]]; then
 sudo yum downgrade -y glibc glibc-common
 sudo yum downgrade -y krb5-libs
 sudo yum downgrade -y libss e2fsprogs-libs e2fsprogs libcom_err
 sudo yum downgrade -y libselinux-utils libselinux-python libselinux
fi

2.2 Install Packages

  • RHEL 7.x

    sudo yum install -y java-1.8.0-openjdk-devel gnutls-devel libaio-devel \
systemtap-sdt-devel lksctp-tools-devel xfsprogs-devel snappy-devel \
libyaml-devel openssl-devel libevent-devel \
gmp-devel mpfr-devel libmpcdec lz4-devel \
libatomic libatomic_ops-devel perl-devel \
automake make git gcc gcc-c++ maven \
unzip bzip2 wget curl xz-devel texinfo \
libffi-devel hwloc-devel libpciaccess-devel libxml2-devel \
libtool diffutils libtool-ltdl-devel trousers-devel \
libunistring-devel libicu-devel readline-devel \
lua-devel patch

  • Ubuntu 16.04

    sudo apt-get update >/dev/null

sudo apt-get install -y openjdk-8-jdk libaio-dev \
 systemtap-sdt-dev lksctp-tools xfsprogs \
 libyaml-dev openssl libevent-dev \
 libmpfr-dev libmpcdec-dev liblz4-dev \
 libssl-dev libsystemd-dev libhwloc-dev \
 libsctp-dev libsnappy-dev libpciaccess-dev libxml2-dev xfslibs-dev \
 libgnutls28-dev libiconv-hook-dev mpi-default-dev libbz2-dev \
 libxslt-dev libjsoncpp-dev ragel \
 libprotobuf-dev protobuf-compiler libcrypto++-dev \
 libtool perl ant libffi-dev \
 automake make git gcc g++ maven ninja-build \
 unzip bzip2 wget curl xz-utils texinfo \
 diffutils liblua5.3-dev libnuma-dev libunistring-dev \
 pigz ragel rapidjson-dev stow

  • Ubuntu 18.04

    sudo apt-get update >/dev/null

sudo apt-get install -y openjdk-8-jdk libaio-dev \
 systemtap-sdt-dev lksctp-tools xfsprogs \
 libyaml-dev openssl libevent-dev \
 libmpfr-dev libmpcdec-dev liblz4-dev \
 libssl1.0-dev libsystemd-dev libhwloc-dev \
 libsctp-dev libsnappy-dev libpciaccess-dev libxml2-dev xfslibs-dev \
 libgnutls28-dev libiconv-hook-dev mpi-default-dev libbz2-dev \
 libxslt-dev libjsoncpp-dev libc-ares-dev ragel \
 libprotobuf-dev protobuf-compiler libcrypto++-dev \
 libtool perl ant libffi-dev \
 automake make git maven ninja-build \
 unzip bzip2 wget curl xz-utils texinfo \
 diffutils gcc-8 g++-8 liblua5.3-dev libnuma-dev libunistring-dev \
 pigz ragel rapidjson-dev stow

3. Build GCC 8 and binutils:

ScyllaDB requires GCC 8 or higher.

  • On RHEL 7.x and Ubuntu 16.04:
  cd "$SOURCE_ROOT"

  URL=http://ftpmirror.gnu.org/binutils/binutils-2.34.tar.gz
  curl -sSL $URL | tar xzf -
  cd binutils-2.34
  mkdir objdir
  cd objdir

  CC=/usr/bin/gcc ../configure --prefix=${PREFIX} --build=s390x-linux-gnu
  make -j 8
  sudo make install


  cd "$SOURCE_ROOT"
  URL=https://ftpmirror.gnu.org/gcc/gcc-8.3.0/gcc-8.3.0.tar.gz
  curl -sSL $URL | tar xzf -

  cd gcc-8.3.0
  ./contrib/download_prerequisites
  mkdir objdir
  cd objdir

  ../configure --enable-languages=c,c++ --prefix=${PREFIX} \
    --enable-shared --enable-threads=posix \
    --disable-multilib --disable-libmpx \
    --with-system-zlib --with-long-double-128 --with-arch=zEC12 \
    --disable-libphobos --disable-werror \
    --build=s390x-linux-gnu --host=s390x-linux-gnu --target=s390x-linux-gnu

  make -j 8 bootstrap
  sudo make install

  export PATH=${PREFIX}/bin${PATH:+:${PATH}}

  LD_LIBRARY_PATH=${PREFIX}/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
  LD_LIBRARY_PATH+=:${PREFIX}/lib
  LD_LIBRARY_PATH+=:/usr/lib64
  export LD_LIBRARY_PATH

  LD_RUN_PATH=${PREFIX}/lib64${LD_RUN_PATH:+:${LD_RUN_PATH}}
  LD_RUN_PATH+=:${PREFIX}/lib
  LD_RUN_PATH+=:/usr/lib64
  export LD_RUN_PATH

  export CC=${PREFIX}/bin/gcc
  export CXX=${PREFIX}/bin/g++
  • On Ubuntu 18.04:
  export PATH=${PREFIX}/bin${PATH:+:${PATH}}

  LD_LIBRARY_PATH=${PREFIX}/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
  LD_LIBRARY_PATH+=:${PREFIX}/lib
  LD_LIBRARY_PATH+=:/usr/lib64
  export LD_LIBRARY_PATH

  LD_RUN_PATH=${PREFIX}/lib64${LD_RUN_PATH:+:${LD_RUN_PATH}}
  LD_RUN_PATH+=:${PREFIX}/lib
  LD_RUN_PATH+=:/usr/lib64
  export LD_RUN_PATH

  export CC=/usr/bin/gcc-8
  export CXX=/usr/bin/g++-8

4. Build Python 3.7

ScyllaDB requires Python 3.7 or higher. The following instructions builds and installs Python 3.7 and the required modules:

  cd "$SOURCE_ROOT"
  URL="https://www.python.org/ftp/python/3.7.4/Python-3.7.4.tgz"
  curl -sSL $URL | tar xzf -
  cd Python-3.7.4
  ./configure
  make
  sudo make install
  pip3 install --user --upgrade pip
  pip3 install --user pyparsing colorama pyyaml

5. Build CMake

The following instructions builds and installs Cmake 3.12:

  cd "$SOURCE_ROOT"
  URL=https://github.com/Kitware/CMake/releases/download/v3.12.4/cmake-3.12.4.tar.gz
  curl -sSL $URL | tar xzf -
  cd cmake-3.12.4
  ./bootstrap
  make
  sudo make install

6 Install additional dependencies:

6.1 Build and install dependencies for RHEL 7.x only:

  • Ninja
  cd "$SOURCE_ROOT"
  curl -sSLO https://github.com/ninja-build/ninja/archive/v1.8.2.zip
  unzip v1.8.2.zip
  cd ninja-1.8.2
  ./configure.py --bootstrap
  sudo cp ninja ${PREFIX}/bin
  • libidn2
  cd ${SOURCE_ROOT}
  URL=https://ftp.gnu.org/gnu/libidn/libidn2-2.3.0.tar.gz
  curl -sSL $URL | tar xzf -
  cd libidn2-2.3.0
  ./configure --disable-doc --disable-gtk-doc
  make
  sudo make install
  • numactl
  cd ${SOURCE_ROOT}
  git clone https://github.com/numactl/numactl.git
  cd numactl
  git checkout 26fba7199c365b55
  ./autogen.sh
  ./configure
  make
  sudo make install
  • c-ares
  cd ${SOURCE_ROOT}
  URL=https://c-ares.haxx.se/download/c-ares-1.14.0.tar.gz
  curl -sSL $URL | tar xzf -
  cd c-ares-1.14.0
  ./configure
  make
  sudo make install
  • Ragel
  cd "$SOURCE_ROOT"
  URL=http://www.colm.net/files/ragel/ragel-6.10.tar.gz
  curl -sSL $URL | tar xzf -
  cd ragel-6.10
  ./configure
  make -j 8
  sudo make install
  • cryptopp
  cd "$SOURCE_ROOT"
  mkdir cryptopp
  cd cryptopp
  curl -sSLO https://www.cryptopp.com/cryptopp565.zip
  unzip cryptopp565.zip
  CXXFLAGS="-std=c++11 -g -O2" make
  sudo make install
  • jsoncpp
  cd "$SOURCE_ROOT"
  URL=https://github.com/open-source-parsers/jsoncpp/archive/1.7.7.tar.gz
  curl -sSL $URL | tar xzf -
  cd jsoncpp-1.7.7
  mkdir -p build/release
  cd build/release
  /usr/local/bin/cmake ../..
  make -j 8
  sudo make install
  • LUA
  cd "$SOURCE_ROOT"
  URL=http://www.lua.org/ftp/lua-5.3.5.tar.gz
  curl -sSL $URL | tar xzf -
  cd lua-5.3.5
  make linux
  sudo make install
  • Protocolbuffers
  cd "$SOURCE_ROOT"
  git clone https://github.com/protocolbuffers/protobuf.git
  cd protobuf
  git checkout v3.10.1
  ./autogen.sh
  ./configure
  make
  sudo make install
  • Ant
  cd "$SOURCE_ROOT"
  URL=https://downloads.apache.org/ant/binaries/apache-ant-1.10.8-bin.tar.gz
  curl -sSL $URL | tar xzf -
  export ANT_HOME="$SOURCE_ROOT/apache-ant-1.10.8"
  export PATH=$PATH:"$ANT_HOME/bin"

6.2 Build and install dependencies for Ubuntu 16.04 only:

  • c-ares
  sudo rm /usr/lib/s390x-linux-gnu/libstdc++.so.6
  sudo ln -s /usr/local/lib64/libstdc++.so.6.0.25 /usr/lib/s390x-linux-gnu/libstdc++.so.6

  cd ${SOURCE_ROOT}
  URL=https://c-ares.haxx.se/download/c-ares-1.14.0.tar.gz
  curl -sSL $URL | tar xzf -
  cd c-ares-1.14.0
  ./configure
  make
  sudo make install

7. Build and install further dependencies

  • Antlr
  cd "$SOURCE_ROOT"

  URL=https://github.com/antlr/antlr3/archive/3.5.2.tar.gz
  curl -sSL $URL | tar xzf -
  cd antlr3-3.5.2
  sudo cp runtime/Cpp/include/antlr3* ${PREFIX}/include/

  cd antlr-complete
  MAVEN_OPTS="-Xmx4G" mvn
  echo 'java -cp '"$(pwd)"'/target/antlr-complete-3.5.2.jar org.antlr.Tool $@' | sudo tee ${PREFIX}/bin/antlr3
  sudo chmod +x ${PREFIX}/bin/antlr3
  • Boost

Set the build environment

On RHEL 7.x and Ubuntu 16.04: TOOLSET=gcc

On Ubuntu 18.04: TOOLSET=gcc-8

  CENV=(PATH=$PATH LD_LIBRARY_PATH=$LD_LIBRARY_PATH LD_RUN_PATH=$LD_RUN_PATH CC=$CC CXX=$CXX)

  cd "$SOURCE_ROOT"
  URL=https://dl.bintray.com/boostorg/release/1.68.0/source/boost_1_68_0.tar.gz
  curl -sSL $URL | tar xzf -
  cd boost_1_68_0

  sed -i 's/array\.hpp/array_wrapper.hpp/g' boost/numeric/ublas/matrix.hpp
  sed -i 's/array\.hpp/array_wrapper.hpp/g' boost/numeric/ublas/storage.hpp

  ./bootstrap.sh

  options=( toolset=$TOOLSET variant=release link=shared runtime-link=shared threading=multi --without-python )

  ./b2 ${options[@]} stage
  sudo ${CENV[@]} ./b2 ${options[@]} install
  • Thrift
  cd "$SOURCE_ROOT"
  URL=http://archive.apache.org/dist/thrift/0.9.3/thrift-0.9.3.tar.gz
  curl -sSL $URL | tar xzf -
  cd thrift-0.9.3
  ./configure --without-java --without-lua --without-go --disable-tests --disable-tutorial
  make -j 8
  sudo make install
  • fmt
  cd "$SOURCE_ROOT"
  git clone https://github.com/fmtlib/fmt.git
  cd fmt
  git checkout d6cea50d01d7779e
  mkdir build
  cd build
  /usr/local/bin/cmake -DFMT_TEST=OFF -DCMAKE_CXX_STANDARD=17 ..
  make
  sudo make install
  • yaml-cpp
  cd "$SOURCE_ROOT"
  URL=https://github.com/jbeder/yaml-cpp/archive/yaml-cpp-0.6.2.tar.gz
  curl -sSL $URL | tar xzf -
  cd yaml-cpp-yaml-cpp-0.6.2
  mkdir build
  cd build
  /usr/local/bin/cmake ..
  make
  sudo make install
  • RapidJSon
  cd "$SOURCE_ROOT"
  git clone https://github.com/Tencent/rapidjson.git
  cd rapidjson
  git checkout v1.1.0
  sudo cp -r ./include/rapidjson ${PREFIX}/include

8. Build ScyllaDB

8.1 Checkout and Patch

The following commands will clone the repository for v3.3.1, initialize its submodules and apply the patches.

  cd "$SOURCE_ROOT"
  git clone https://github.com/scylladb/scylla.git
  cd scylla
  git checkout scylla-3.3.1
  git submodule update --init --recursive

  URL=https://raw.githubusercontent.com/linux-on-ibm-z/scripts/master/ScyllaDB/3.3.1/patch/                                                     

  curl -sSL ${URL}/seastar.diff | patch -d seastar -p1 || echo "Error"
  curl -sSL ${URL}/scylla.diff | patch -p1 || echo "Error"

8.2 Configure and compile ScyllaDB

The following commands will build a release binary that is optimized for, and will only run, on the specified TARGET:

  export PKG_CONFIG_PATH=$PREFIX/lib/pkgconfig:${PKG_CONFIG_PATH}

  cflags="-I${PREFIX}/include -I${PREFIX}/include/boost -L${PREFIX}/lib -L${PREFIX}/lib64 "

  ./configure.py --mode release --target ${TARGET} --debuginfo 1 \
    --static-thrift --cflags "${cflags}" --ldflags="-Wl,--build-id=sha1" \
    --compiler "${CXX}" --c-compiler "${CC}"

  ninja -j 8

9. Test ScyllaDB

Once ScyllaDB has been built you can run the tests. By default the tests will not contain debug information. To enable debug information add --tests-debuginfo 1 to the configure.py parameters above. This will make the build significantly larger (10s of gigabytes).

  cd "$SOURCE_ROOT/scylla"
  ./test.py --mode release

10. Performance Tuning

The performance tuning scripts that ship with ScyllaDB don't fully support LinuxONE at this stage. To maximize performance receive packet steering (RPS) should be enabled on network adapters.

11. Run ScyllaDB

The scylla binary can be found at /<source_root>/scylla/build/release/scylla.

$SOURCE_ROOT/scylla/build/release/scylla --help

For more information on how to run and test ScyllaDB see the links below.

References

The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue or contact us on IBM Z Community if you have any questions or feedback.

Clone this wiki locally