Building Apache Storm

Apache Storm

The instructions provided below specify the steps to build Apache Storm version 1.0.3 on Linux on z Systems for the following distributions:

• RHEL (7.1, 7.2, 7.3)
• SLES (12 SP1, 12 SP2)
• Ubuntu (16.04, 16.10)

General Notes:

• When following the steps below please use a standard permission user unless otherwise specified.
• A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Step 1: Building and installing Apache Storm

1.1) Install Dependencies

• RHEL (7.1, 7.2, 7.3)

  sudo yum install git tar java-1.8.0-openjdk-devel.s390x  

• SLES (12 SP1, 12 SP2)

  sudo zypper install git java-1_8_0-openjdk tar  

• Ubuntu (16.04, 16.10)

  sudo apt-get install git tar ant wget openjdk-8-jdk maven curl gpgv nodejs gnupg2

1.2) Install Maven(For RHEL and SLES only)

Maven v3.2.5 or below is required. There is a bug in later versions of maven as linked to from https://issues.apache.org/jira/browse/MSHADE-206 that cause shaded dependencies to not be packaged correctly.

Instructions for building Maven can be found here. Replace v3.3.x source tarball with v3.2.5: https://archive.apache.org/dist/maven/maven-3/3.2.5/source/apache-maven-3.2.5-src.tar.gz.

1.3) Install NodeJS SDK (For RHEL and SLES only)

Install nodejs sdk from https://developer.ibm.com/node/sdk/ for Linux on System z 64-bit

Add the nodejs executable to the path:

  export PATH=$PATH:<Nodejs-Install-Directory>/bin  

1.4) Install rvm and nvm

curl -L https://get.rvm.io | bash -s stable --autolibs=enabled && source $HOME/.profile  
command curl -sSL https://rvm.io/mpapis.asc | gpg2 --import -  
wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.26.1/install.sh | bash && source $HOME/.bashrc

rvm use 2.1.5 --install
nvm use system

1.5) Set environment variables

Setup up Maven and JAVA:

export JAVA_HOME=/usr/lib/jvm/java  #for RHEL
export JAVA_HOME=/usr/lib64/jvm/java  #for SLES
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-s390x #for Ubuntu
export M2_HOME=<Your-Maven-location> #for RHEL and SLES only
export PATH=$PATH:$M2_HOME/bin #for RHEL and SLES only
export PATH=$JAVA_HOME/bin:$PATH
export _JAVA_OPTIONS=-Xmx4096m
export JVM_ARGS="-Xms4096m -Xmx4096m"

1.6) Download the source code

cd /<source_root>/
git clone https://github.com/apache/storm.git  
cd storm  
git checkout tags/v1.0.3

1.7) Build Apache Storm

mvn clean install

Note: If you wish to skip the unit tests you can do this by adding -DskipTests to the command line.

Step 2: Testing(Optional)

2.1) Run unit test cases

To run Clojure and Java unit tests but no integration tests execute the command

cd /<source_root>/storm
mvn test

Note: Integration tests require that you activate the profile integration-test and that you specify the maven-failsafe-plugin in the module pom file.

Step 3: Create a Storm distribution (packaging)

cd /<source_root>/storm/storm-dist/binary
mvn package  

Note:

• After running mvn package you may be asked to enter your GPG/PGP credentials (once for each binary file, in fact). This happens because the packaging step will create *.asc digital signatures for all the binaries, and in the workflow above your GPG private key will be used to create those signatures.
• The binaries can be found at /<source_root>/storm/storm-dist/binary/target/.

References:

• http://storm.apache.org/releases/current/index.html
• https://github.com/apache/storm/blob/v1.0.3/DEVELOPER.md