Skip to content

Building ANTLR 4

Jump to bottom
aborkar-ibm edited this page Jan 28, 2021 · 42 revisions

Building ANTLR v4.9.1

Below versions of ANTLR (ANother Tool for Language Recognition) version 4 are available in respective distributions at the time of creation of these build instructions:

  • Ubuntu (18.04, 20.04, 20.10) has ANTLR 4.7.2-2

If only the Java runtime is required, then the pre-built jars for ANTLR v4.9.1 can be downloaded from here.

The instructions provided below specify the steps to build ANTLR v4.9.1 on Linux on IBM Z for the following distributions:

  • RHEL (7.8, 7.9, 8.1, 8.2, 8.3)
  • Ubuntu (18.04, 20.04, 20.10)
  • SLES (12 SP5, 15 SP2)

The instructions below describe:

  1. How to download and install the required Java SDK.
  2. If only the ANTLR Java runtime is required - how to download and install ANTLR pre-built binaries
  3. If other ANTLR runtimes are required - how to download and install the required packages, build ANTLR and the supported runtimes from source.

General Notes:

  • When following the steps below please use a standard permission user unless otherwise specified.
  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Step 1: Install Java SDK

1.1) Download and install Java

Download and install AdoptOpenJDK 11 (HotSpot or Eclipse OpenJ9) from here.

Note: At the time of creation of these build instructions, ANTLR was verified with AdoptOpenJDK 11 with HotSpot ((build 11.0.9.1+1) and AdoptOpenJDK 11 with OpenJ9 (build 11.0.9+11).

1.2) Set Environment variables

export SOURCE_ROOT=/<source_root>/
export JAVA_HOME=<path to installed java>
export PATH=$JAVA_HOME/bin:$PATH

Step 2: If only the ANTLR Java runtime is required.

2.1) Download and install ANTLR pre-built binaries.

cd $SOURCE_ROOT
curl -s -S -L -O https://www.antlr.org/download/antlr-4.9.1-complete.jar
JAR=$SOURCE_ROOT/antlr-4.9.1-complete.jar
export CLASSPATH=".:$JAR:$CLASSPATH"

2.2) Test the ANTLR tool and Java runtime

Test the tool and runtime using the example provided here

alias antlr4='java -Xmx500M -cp "$JAR:$CLASSPATH" org.antlr.v4.Tool'
alias grun='java org.antlr.v4.gui.TestRig'

cat > Hello.g4 <<'EOF'
  grammar Hello;
  r  : 'hello' ID ;
  ID : [a-z]+ ;
  WS : [ \t\r\n]+ -> skip ;
EOF

antlr4 Hello.g4
javac Hello*.java
diff -w <(grun Hello r -tree <<<"hello world") <(echo "(r hello world)")

Step 3: If other Antlr runtimes are required.

3.1) Install the required packages

  • RHEL (7.8, 7.9)
sudo yum install -y unzip xz libuuid-devel curl wget git make devtoolset-7-gcc devtoolset-7-gcc-c++ llvm-toolset-7-clang python2 golang
  • RHEL (8.1, 8.2, 8.3)
sudo yum install -y unzip xz libuuid-devel curl wget git make gcc gcc-c++ python2 python3.8 golang cmake clang
  • UBUNTU (18.04, 20.04, 20.10)
sudo apt-get update
sudo apt-get install -y unzip xz-utils uuid-dev curl wget git make python python3.8 golang gcc g++ cmake clang
  • SLES 12 SP5
sudo zypper install -y unzip tar xz libuuid-devel curl wget git make gcc8 gcc8-c++ python python-typing cmake
  • SLES 15 SP2
sudo zypper install -y unzip xz libuuid-devel curl wget git make gcc9 gcc9-c++ python go1.15 cmake clang7

3.2) Download(or Install) required binaries(packages)

  • GO (Only for SLES 12 SP5): Follow the instructions to install GO

  • Install Python 3.8.x (Only for RHEL 7.x and SLES)

    • RHEL (7.8, 7.9)
    sudo yum install -y bzip2-devel gcc gcc-c++ gdbm-devel libdb-devel libffi-devel libuuid-devel make ncurses-devel readline-devel sqlite-devel tar tk-devel wget xz xz-devel zlib-devel
    • SLES 12 SP5
    sudo zypper install -y gawk gcc gcc-c++ gdbm-devel libbz2-devel libdb-4_8-devel libffi48-devel libuuid-devel make ncurses-devel readline-devel sqlite3-devel tar tk-devel wget xz-devel zlib-devel
    • SLES 15 SP2
    sudo zypper install -y gawk gcc gcc-c++ gdbm-devel libbz2-devel libdb-4_8-devel libffi-devel libnsl-devel libopenssl-devel libuuid-devel make ncurses-devel readline-devel sqlite3-devel tar tk-devel wget xz-devel zlib-devel gzip
    • Build and install OpenSSL from source (SLES 12 SP5 and RHEL 7.x only)
    cd $SOURCE_ROOT
wget https://www.openssl.org/source/openssl-1.1.1h.tar.gz
tar -xzvf openssl-1.1.1h.tar.gz
cd openssl-1.1.1h
./config --prefix=/usr/local --openssldir=/usr/local
make
sudo make install
export PATH=/usr/local/bin:$PATH

export LDFLAGS="-L/usr/local/lib/ -L/usr/local/lib64/"
export LD_LIBRARY_PATH="/usr/local/lib/ /usr/local/lib64/ /usr/lib/ /usr/lib64/"
export CPPFLAGS="-I/usr/local/include/ -I/usr/local/include/openssl"
    • Install python
    cd $SOURCE_ROOT
wget https://www.python.org/ftp/python/3.8.6/Python-3.8.6.tgz
tar -xzf Python-3.8.6.tgz
cd Python-3.8.6
./configure 
make && sudo make install
python3 -V

  • Install cmake (Only for RHEL 7.x)

cd $SOURCE_ROOT
wget https://cmake.org/files/v3.5/cmake-3.5.2.tar.gz
tar -xzf cmake-3.5.2.tar.gz
cd cmake-3.5.2
./bootstrap
make
sudo make install -e LD_LIBRARY_PATH=/usr/local/lib64/

  • Install Clang 8.0.1 (Only SLES 12SP5)

    • Build and Install LLVM Dependency
    export CC=gcc-8
export CXX=g++-8

cd $SOURCE_ROOT
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-8.0.1/llvm-8.0.1.src.tar.xz
tar -xvf cfe-8.0.1.src.tar.xz
cd cfe-8.0.1.src/
mkdir build && cd build
cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=release ..
make && sudo make install
    • Build and Install CLANG
    cd $SOURCE_ROOT
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-8.0.1/cfe-8.0.1.src.tar.xz
tar -xvf cfe-8.0.1.src.tar.xz
cd cfe-8.0.1.src/
mkdir build && cd build
cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=release ..
make && sudo make install

  • Download NodeJS, Maven and ANTLR source for all distros.

cd $SOURCE_ROOT

TARBALL=node-v14.15.1-linux-s390x.tar.xz
curl -s -S -L -O https://nodejs.org/dist/v14.15.1/${TARBALL}
tar xJf ${TARBALL}

TARBALL=apache-maven-3.3.9-bin.tar.gz
curl -s -S -L -O https://archive.apache.org/dist/maven/maven-3/3.3.9/binaries/${TARBALL}
tar xzf ${TARBALL}

ZIP=antlr4-4.9.1.zip
curl -s -S -L -o ${ZIP} https://github.com/antlr/antlr4/archive/4.9.1.zip
unzip ${ZIP}

3.3) Build ANTLR and the supported runtimes

Set the PATH and compiler environment variables.

  • RHEL (7.8, 7.9)
export PATH=/opt/rh/devtoolset-7/root/usr/bin:$JAVA_HOME/bin:$SOURCE_ROOT/node-v14.15.1-linux-s390x/bin:$SOURCE_ROOT/apache-maven-3.3.9/bin:${PATH}
export CC=gcc
export CXX=g++
  • RHEL (8.1, 8.2, 8.3) and UBUNTU (18.04, 20.04, 20.10)
export PATH=$JAVA_HOME/bin:$SOURCE_ROOT/node-v14.15.1-linux-s390x/bin:$SOURCE_ROOT/apache-maven-3.3.9/bin:${PATH}
export CC=gcc
export CXX=g++
  • SLES 12 SP5
export PATH=$JAVA_HOME/bin:$SOURCE_ROOT/node-v14.15.1-linux-s390x/bin:$SOURCE_ROOT/apache-maven-3.3.9/bin:${PATH}
export CC=gcc-8
export CXX=g++-8
  • SLES 15 SP2
export PATH=$JAVA_HOME/bin:$SOURCE_ROOT/node-v14.15.1-linux-s390x/bin:$SOURCE_ROOT/apache-maven-3.3.9/bin:${PATH}
export CC=gcc-9
export CXX=g++-9

Build the ANTLR tool, then build the supported runtimes.

export MAVEN_OPTS="-Xmx1G"

cd $SOURCE_ROOT/antlr4-4.9.1
mvn install -DskipTests=true
cd runtime/Cpp
mkdir build && mkdir run
cd build
cmake -DWITH_LIBCXX=Off -DCMAKE_BUILD_TYPE=release ..
make -j 8
DESTDIR=$SOURCE_ROOT/antlr4-4.9.1/runtime/Cpp/run make install

Note: If multiple gcc versions are installed, DCMAKE_CXX_COMPILER and DCMAKE_C_COMPILER flags can be used in cmake command to give correct version path. Note: ANTLR 4.9.1 runtimes supported for Linux on IBM Z are Java, Python, Go, NodeJS and C++.

Step 4: Test (Optional)

cd $SOURCE_ROOT/antlr4-4.9.1/runtime-testsuite
mvn -Dtest=java.* test
mvn -Dtest=python2.* test
mvn -Dtest=python3.* test
mvn -Dtest=go.* test
mvn -Dtest=javascript.* test
mvn -Dtest=cpp.* test

References:

http://www.antlr.org
https://github.com/antlr/antlr4/blob/master/doc/getting-started.md
https://github.com/antlr/antlr4/blob/master/doc/antlr-project-testing.md
https://github.com/antlr/antlr4/blob/master/doc/targets.md

The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue or contact us on IBM Z Community if you have any questions or feedback.

Clone this wiki locally