Skip to content

Building ANTLR 4

Jump to bottom
aborkar-ibm edited this page May 20, 2020 · 42 revisions

Building ANTLR v4.8

Below versions of ANTLR (ANother Tool for Language Recognition) version 4 are available in respective distributions at the time of creation of these build instructions:

  • Ubuntu 16.04 have ANTLR 4.5.1
  • Ubuntu (18.04, 20.04) has ANTLR 4.7.2

If only the Java runtime is required, then the pre-built jars for ANTLR v4.8 can be downloaded from here.

The instructions provided below specify the steps to build ANTLR v4.8 on Linux on IBM Z for the following distributions:

  • RHEL (7.6, 7.7, 7.8, 8.1, 8.2)
  • Ubuntu (16.04, 18.04, 20.04)
  • SLES (12 SP4, 12 SP5, 15 SP1)

The instructions below describe:

  1. How to download and install the required Java SDK.
  2. If only the ANTLR Java runtime is required - how to download and install ANTLR pre-built binaries
  3. If other ANTLR runtimes are required - how to download and install the required packages, build ANTLR and the supported runtimes from source.

General Notes:

  • When following the steps below please use a standard permission user unless otherwise specified.
  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Step 1: Install Java SDK

1.1) Download and install Java

Download and install AdoptOpenJDK 11 (HotSpot or Eclipse OpenJ9) from here.

1.2) Set Environment variables

export SOURCE_ROOT=/<source_root>/
export JAVA_HOME=<path to installed java>
export PATH=$JAVA_HOME/bin:$PATH

Step 2: If only the ANTLR Java runtime is required.

2.1) Download and install ANTLR pre-built binaries.

cd $SOURCE_ROOT
curl -s -S -L -O https://www.antlr.org/download/antlr-4.8-complete.jar
JAR=$SOURCE_ROOT/antlr-4.8-complete.jar
export CLASSPATH=".:$JAR:$CLASSPATH"

2.2) Test the ANTLR tool and Java runtime

Test the tool and runtime using the example provided here

antlr4() { java -Xmx500M -cp "$JAR:$CLASSPATH" org.antlr.v4.Tool $*; }
grun() { java -Xmx500M -cp "$JAR:$CLASSPATH" org.antlr.v4.gui.TestRig $*; }

cat > Hello.g4 <<'EOF'
  grammar Hello;
  r  : 'hello' ID ;
  ID : [a-z]+ ;
  WS : [ \t\r\n]+ -> skip ;
EOF

antlr4 Hello.g4
javac Hello*.java
diff -w <(grun Hello r -tree <<<"hello world") <(echo "(r hello world)")

Step 3: If other Antlr runtimes are required.

3.1) Install the required packages

  • RHEL (7.6, 7.7, 7.8)
sudo yum install -y unzip xz libuuid-devel curl wget git make devtoolset-8-gcc devtoolset-8-gcc-c++ python2 python3 golang cmake patch
  • RHEL (8.1, 8.2)
sudo yum install -y unzip xz libuuid-devel curl wget git make gcc gcc-c++ python2 python3 golang cmake patch
  • UBUNTU (16.04, 18.04, 20.04)
sudo apt-get update
sudo apt-get install -y unzip xz-utils uuid-dev curl wget git make python python3 golang gcc g++ cmake
  • SLES (12 SP4, 12 SP5)
sudo zypper install -y unzip tar xz libuuid-devel curl wget git make gcc8 gcc8-c++ python python3 python3-pip cmake
sudo pip3 install typing
  • SLES (15 SP1)
sudo zypper install -y unzip xz libuuid-devel curl wget git make gcc8 gcc8-c++ python python3 python3-pip cmake patch

3.2) Download required binaries

  • GO (Only for SLES): Follow the instructions to install GO

  • Download NodeJS, Maven and ANTLR source for all distros.

cd $SOURCE_ROOT

TARBALL=node-v12.14.1-linux-s390x.tar.xz
curl -s -S -L -O https://nodejs.org/dist/v12.14.1/${TARBALL}
tar xJf ${TARBALL}

TARBALL=apache-maven-3.3.9-bin.tar.gz
curl -s -S -L -O https://archive.apache.org/dist/maven/maven-3/3.3.9/binaries/${TARBALL}
tar xzf ${TARBALL}

ZIP=antlr4-4.8.zip
curl -s -S -L -o ${ZIP} https://github.com/antlr/antlr4/archive/4.8.zip
unzip ${ZIP}

3.3) Build ANTLR and the supported runtimes

On RHEL and UBUNTU, use the g++ compiler rather than clang++.

cd $SOURCE_ROOT/antlr4-4.8

patch --unified -p0 \
runtime-testsuite/test/org/antlr/v4/test/runtime/cpp/BaseCppTest.java << EOF
@@ -550,7 +550,7 @@
 		System.out.println("Building ANTLR4 C++ runtime (if necessary) at "+ runtimePath);
 
 		try {
-			String command[] = { "cmake", ".", /*"-DCMAKE_CXX_COMPILER=clang++",*/ "-DCMAKE_BUILD_TYPE=release" };
+			String command[] = { "cmake", ".", /*"-DCMAKE_CXX_COMPILER=g++",*/ "-DCMAKE_BUILD_TYPE=release" };
 			if (runCommand(command, runtimePath, "antlr runtime cmake", false) == null) {
 				return false;
 			}
@@ -604,7 +604,7 @@
 		synchronized (runtimeBuiltOnce) {
 			if ( !runtimeBuiltOnce ) {
 				try {
-					String command[] = {"clang++", "--version"};
+					String command[] = {"g++", "--version"};
 					String output = runCommand(command, tmpdir, "printing compiler version", false);
 					System.out.println("Compiler version is: "+output);
 				}
@@ -635,8 +635,9 @@
 		}
 
 		try {
-			List<String> command2 = new ArrayList<String>(Arrays.asList("clang++", "-std=c++11", "-I", includePath, "-L.", "-lantlr4-runtime", "-o", "a.out"));
+			List<String> command2 = new ArrayList<String>(Arrays.asList("g++", "-std=c++11", "-I", includePath, "-L.", "-o", "a.out"));
 			command2.addAll(allCppFiles(tmpdir));
+                        command2.add("-lantlr4-runtime");
 			if (runCommand(command2.toArray(new String[0]), tmpdir, "building test binary", true) == null) {
 				return null;
 			}
EOF

On SLES use the g++8 compiler rather than clang++

cd $SOURCE_ROOT/antlr4-4.8

patch --unified -p0 \
runtime-testsuite/test/org/antlr/v4/test/runtime/cpp/BaseCppTest.java << EOF
@@ -550,7 +550,7 @@
 		System.out.println("Building ANTLR4 C++ runtime (if necessary) at "+ runtimePath);
 
 		try {
-			String command[] = { "cmake", ".", /*"-DCMAKE_CXX_COMPILER=clang++",*/ "-DCMAKE_BUILD_TYPE=release" };
+			String command[] = { "cmake", ".", /*"-DCMAKE_CXX_COMPILER=g++",*/ "-DCMAKE_BUILD_TYPE=release" };
 			if (runCommand(command, runtimePath, "antlr runtime cmake", false) == null) {
 				return false;
 			}
@@ -604,7 +604,7 @@
 		synchronized (runtimeBuiltOnce) {
 			if ( !runtimeBuiltOnce ) {
 				try {
-					String command[] = {"clang++", "--version"};
+					String command[] = {"g++-8", "--version"};
 					String output = runCommand(command, tmpdir, "printing compiler version", false);
 					System.out.println("Compiler version is: "+output);
 				}
@@ -635,8 +635,9 @@
 		}
 
 		try {
-			List<String> command2 = new ArrayList<String>(Arrays.asList("clang++", "-std=c++11", "-I", includePath, "-L.", "-lantlr4-runtime", "-o", "a.out"));
+			List<String> command2 = new ArrayList<String>(Arrays.asList("g++-8", "-std=c++11", "-I", includePath, "-L.", "-o", "a.out"));
 			command2.addAll(allCppFiles(tmpdir));
+                        command2.add("-lantlr4-runtime");
 			if (runCommand(command2.toArray(new String[0]), tmpdir, "building test binary", true) == null) {
 				return null;
 			}
EOF

On all distros, use gcc stdc++ rather than clang libc++ and relax the requirements on the Python version for unit testing.

cd $SOURCE_ROOT/antlr4-4.8

patch --unified -p0 runtime/Cpp/CMakeLists.txt <<EOF
@@ -19,7 +19,7 @@
     FORCE)
 endif(NOT WITH_DEMO)
 
-option(WITH_LIBCXX "Building with clang++ and libc++(in Linux). To enable with: -DWITH_LIBCXX=On" On)
+option(WITH_LIBCXX "Building with clang++ and libc++(in Linux). To enable with: -DWITH_LIBCXX=On" Off)
 option(WITH_STATIC_CRT "(Visual C++) Enable to statically link CRT, which avoids requiring users to install the redistribution package.
  To disable with: -DWITH_STATIC_CRT=Off" On)
 
EOF

patch --unified -p0 \
runtime-testsuite/test/org/antlr/v4/test/runtime/python3/\
BasePython3Test.java <<EOF
@@ -19,7 +19,7 @@
 
 	@Override
 	protected String getPythonExecutable() {
-		return "python3.7";
+		return "python3";
 	} // force 3.7
 
 	@Override
EOF


patch --unified -p0 \
runtime-testsuite/test/org/antlr/v4/test/runtime/python2/\
BasePython2Test.java <<EOF
@@ -20,7 +20,7 @@
 
 	@Override
 	protected String getPythonExecutable() {
-		return "python2.7";
+		return "python2";
 	}
 
 	@Override
EOF

Set the PATH and compiler environment variables.

  • RHEL (7.6, 7.7, 7.8)
export PATH=/opt/rh/devtoolset-8/root/usr/bin:$JAVA_HOME/bin:$SOURCE_ROOT/node-v12.14.1-linux-s390x/bin:$SOURCE_ROOT/apache-maven-3.3.9/bin:${PATH}
export CC=gcc
export CXX=g++
  • RHEL (8.1, 8,2) and UBUNTU (16.04, 18.04, 20.04)
export PATH=$JAVA_HOME/bin:$SOURCE_ROOT/node-v12.14.1-linux-s390x/bin:$SOURCE_ROOT/apache-maven-3.3.9/bin:${PATH}
export CC=gcc
export CXX=g++
  • SLES (12 SP4, 12 SP5, 15 SP1)
export PATH=$JAVA_HOME/bin:$SOURCE_ROOT/node-v12.14.1-linux-s390x/bin:$SOURCE_ROOT/apache-maven-3.3.9/bin:${PATH}
export CC=gcc-8
export CXX=g++-8

Build the ANTLR tool, then build the supported runtimes.

export MAVEN_OPTS="-Xmx1G"

cd $SOURCE_ROOT/antlr4-4.8
mvn install -DskipTests=true
cd runtime/Cpp
mkdir build && mkdir run
cd build
cmake -DWITH_LIBCXX=Off -DCMAKE_BUILD_TYPE=release ..
make -j 8
DESTDIR=$SOURCE_ROOT/antlr4-4.8/runtime/Cpp/run make install

Note: ANTLR 4.8 runtimes are supported for Linux on IBM Z are Java, Python, Go, NodeJS and C++.

Step 4: Test (Optional)

cd $SOURCE_ROOT/antlr4-4.8/runtime-testsuite
mvn -Dtest=java.* test
mvn -Dtest=python2.* test
mvn -Dtest=python3.* test
mvn -Dtest=go.* test
mvn -Dtest=node.* test
mvn -Dtest=cpp.* test

References:

http://www.antlr.org
https://github.com/antlr/antlr4/blob/master/doc/getting-started.md
https://github.com/antlr/antlr4/blob/master/doc/antlr-project-testing.md
https://github.com/antlr/antlr4/blob/master/doc/targets.md

The information provided in this article is accurate at the time of writing, but on-going development in the open-source projects involved may make the information incorrect or obsolete. Please open issue or contact us on IBM Z Community if you have any questions or feedback.

Clone this wiki locally