Skip to content

Building ANTLR 4

aborkar-ibm edited this page Jan 17, 2020 · 42 revisions

Building ANTLR v4

Below versions of ANTLR (ANother Tool for Language Recognition) v4 are available in respective distributions at the time of creation of these build instructions:

  • Ubuntu 16.04 has 4.5.1
  • Ubuntu (18.04, 19.10) has 4.7.2

The pre-built jars for ANTLR version 4.7.2 can be downloaded from here. Install dependencies as mentioned in Step 1 for the Java runtime target. For more information, see getting started and targets.md .

The instructions provided below specify the steps to build ANTLR v4 version 4.7.2 on Linux on IBM Z for the following distributions:

  • RHEL (7.5, 7.6, 7.7)
  • SLES (12 SP4, 15 SP1)
  • Ubuntu 16.04

General Notes:

  • When following the steps below please use a standard permission user unless otherwise specified.
  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Step 1: Obtain dependencies

1.1) Install the dependencies

export SOURCE_ROOT=/<source_root>/
  • RHEL (7.5, 7.6, 7.7)

    • With IBM-SDK

      sudo yum install -y tar python wget git java-1.8.0-ibm-devel ant
      Set environment variables
      export JAVA_HOME=/usr/lib/jvm/java-1.8.0-ibm
      export PATH=$JAVA_HOME/bin:$PATH
    • With OpenJDK

      sudo yum install -y tar python wget git java-1.8.0-openjdk-devel ant
      Set environment variables
      export JAVA_HOME=/usr/lib/jvm/java-1.8.0
      export PATH=$JAVA_HOME/bin:$PATH
    • Build and install GCC 4.9.4 for Node.js

      sudo yum install -y make flex gcc gcc-c++ binutils-devel bzip2
      cd $SOURCE_ROOT
      wget ftp://gcc.gnu.org/pub/gcc/releases/gcc-4.9.4/gcc-4.9.4.tar.gz
      tar -xzf gcc-4.9.4.tar.gz && cd gcc-4.9.4/
      ./contrib/download_prerequisites && cd ..
      mkdir gccbuild && cd gccbuild
      ../gcc-4.9.4/configure --prefix=$HOME/install/gcc-4.9.4 --enable-checking=release --enable-languages=c,c++ \
      --disable-multilib --enable-shared --enable-threads=posix --with-system-zlib
      make && sudo make install
      export PATH=$HOME/install/gcc-4.9.4/bin:$PATH
      export LD_LIBRARY_PATH=$HOME/install/gcc-4.9.4/lib64/
      gcc --version
  • SLES (12 SP4, 15 SP1)

    • With IBM-SDK

      sudo zypper install tar python wget git java-1_8_0-ibm-devel ant git xz
      Set environment variables
      export JAVA_HOME=/usr/lib64/jvm/java-1.8.0-ibm
      export PATH=$JAVA_HOME/bin:$PATH
    • With OpenJDK

      sudo zypper install tar python wget git java-1_8_0-openjdk-devel ant git xz
      Set environment variables
      export JAVA_HOME=/usr/lib64/jvm/java-1.8.0
      export PATH=$JAVA_HOME/bin:$PATH
  • Ubuntu 16.04

    • With IBM-SDK

      sudo apt-get update
      sudo apt-get install -y tar python nodejs golang maven wget git ant patch

      To install IBM Java 8, download IBM Java 8 SDK binary from IBM Java 8 and follow the instructions as per given in the link. Update JAVA_HOME and PATH accordingly.

    • With OpenJDK

      sudo apt-get update
      sudo apt-get install -y tar python nodejs golang maven wget git ant patch openjdk-8-jdk
      Set environment variables
      export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-s390x/
      export PATH=$JAVA_HOME/bin:$PATH
    • Create a symbolic link to nodejs

      sudo ln -s `which nodejs` /usr/local/bin/node

1.2) Build Mono 4.8.0

  • Install dependencies

    • RHEL (7.5, 7.6, 7.7)

      sudo yum install -y autoconf automake libtool which ncurses patch
    • SLES (12 SP4, 15 SP1)

      sudo zypper install -y autoconf automake libtool which gcc-c++ make ncurses patch timezone
    • Ubuntu 16.04

      sudo apt-get install -y autoconf automake libtool make libncurses-dev tzdata

    Note: There is issue with the available GCC version on Ubuntu and SLES 15 SP1, to build mono you need to build GCC 6.3.0 from source.

    Following are the instructions to build GCC 6.3.0 from source:

    • Install dependencies:

      • Ubuntu 16.04

        sudo apt-get install -y make flex gcc g++ binutils-dev bzip2 libc6-dev-s390 gcc-multilib g++-multilib libc6-dev libgmp3-dev libmpfr-dev libmpc-dev libz-dev
      • SLES 15 SP1

        sudo zypper install -y flex zlib-devel glibc-devel-32bit bzip2 gzip gawk
    • Download source

      cd $SOURCE_ROOT
      git clone https://github.com/gcc-mirror/gcc
      cd gcc
      git checkout releases/gcc-6.3.0
      ./contrib/download_prerequisites
    • Patch GCC 6.3.0

      diff --git a/gcc/ubsan.c b/gcc/ubsan.c
      index 56637d8f0a9..109382498fd 100644
      --- a/gcc/ubsan.c
      +++ b/gcc/ubsan.c
      @@ -1471,7 +1471,7 @@ ubsan_use_new_style_p (location_t loc)
      
         expanded_location xloc = expand_location (loc);
         if (xloc.file == NULL || strncmp (xloc.file, "\1", 2) == 0
      -      || xloc.file == '\0' || xloc.file[0] == '\xff'
      +      || xloc.file[0] == '\0' || xloc.file[0] == '\xff'
             || xloc.file[1] == '\xff')
           return false;
      
    • Configure, build and install GCC 6.3.0

      mkdir -p ../gccbuild
      cd ../gccbuild
      echo $LIBRARY_PATH
      ../gcc/configure \
      --prefix="/opt/gcc" \
      --enable-shared --with-system-zlib --enable-threads=posix \
      --enable-__cxa_atexit --enable-checking --enable-gnu-indirect-function \
      --enable-languages="c,c++" --disable-bootstrap --disable-multilib
      make all
      sudo make install
      export PATH=/opt/gcc/bin:$PATH
      export LD_LIBRARY_PATH='/opt/gcc/$LIB'
  • Download Mono 4.8.0 source

    cd $SOURCE_ROOT
    git clone git://github.com/mono/mono
    cd mono
    git checkout mono-4.8.0.524

    Note: git checkout may fail for RHEL and SLES distros while switching between the branches, use -f option to avoid the errors.

    Patch the following file:

    diff --git a/mono/io-layer/processes.c b/mono/io-layer/processes.c
      index 53c271a4493..36f4c57adf1 100644
    --- a/mono/io-layer/processes.c
    +++ b/mono/io-layer/processes.c
    @@ -18,6 +18,7 @@
     #include <sys/time.h>
     #include <errno.h>
     #include <sys/types.h>
    +#include <sys/sysmacros.h>
     #include <sys/stat.h>
     #include <unistd.h>
     #ifdef HAVE_SIGNAL_H

    Create a patch file patch_context.diff with the following contents:

    diff --git a/mono/utils/mono-context.h b/mono/utils/mono-context.h
    index 68a1209..6cb080a 100644
    --- a/mono/utils/mono-context.h
    +++ b/mono/utils/mono-context.h
    @@ -854,7 +854,11 @@ typedef struct {
    
     #define MONO_ARCH_HAS_MONO_CONTEXT 1
    
    +#if __GLIBC_PREREQ(2, 26)
    +typedef ucontext_t MonoContext;
    +#else
     typedef struct ucontext MonoContext;
    +#endif
    
     #define MONO_CONTEXT_SET_IP(ctx,ip)                                    \
            do {                                                            \

    Apply patch using command mention below:

    patch mono/utils/mono-context.h < patch_context.diff
    
  • Build Mono 4.8.0

    ./autogen.sh --disable-nls
    make get-monolite-latest
    make
    sudo make install

1.3) Install Maven 3.3.9 (RHEL and SLES only)

cd $SOURCE_ROOT
wget https://archive.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
tar -xzf apache-maven-3.3.9-bin.tar.gz
export PATH=$SOURCE_ROOT/apache-maven-3.3.9/bin::$PATH

1.4) Install Node.js (RHEL 7 and SLES only)

Download the latest version of Node.js for Linux on System z from their official website and install to your system. Alternatively you can install IBM SDK for Node.js.

1.5) Build Python 2.7 and 3.6

Python 2.7 and 3.6 are required. Necessary instructions to build Python 2.7.x and 3.6.x can be found here.

For RHEL 7.7, install Python v3.6.* with yum install python3.

For SLES (15 SP1), install Python v3.6.* with zypper install python3.

1.6) Install Go (RHEL and SLES only)

Instructions to install Go can be found here.

Step 2: Build and install ANTLR v4

2.1) Download and apply patch to source code

cd $SOURCE_ROOT
git clone git://github.com/antlr/antlr4.git
cd antlr4
git checkout 4.7.2
export MAVEN_OPTS="-Xmx1G"

Apply the following patch:

diff --git a/runtime-testsuite/pom.xml b/runtime-testsuite/pom.xml
index dc33587..2b8d461 100644
--- a/runtime-testsuite/pom.xml
+++ b/runtime-testsuite/pom.xml
@@ -68,7 +68,8 @@
                <dependency>
                  <groupId>org.eclipse.jetty</groupId>
                  <artifactId>jetty-server</artifactId>
-                 <version>[9.3.24.v20180605,)</version>
+                 <!-- stick to version 9 for jdk compatibility -->
+                 <version>9.4.19.v20190610</version>
                  <scope>test</scope>
                </dependency>
                <dependency>

2.3) Build and test ANTLR4

mvn install

Note: Some tests may fail due to timeout, (e.g. org.antlr.v4.test.tool.TestPerformance) in-case increase the timeout in the respective test class and re-run the build.

References:

http://www.antlr.org
https://github.com/antlr/antlr4/blob/master/doc/antlr-project-testing.md

Clone this wiki locally