UZH Advanced Topics of Artificial Intelligence (ATAI) Fall 23

This project is based on SparkNLP.

Installations

It's highly recommended to run this project using macOS or Linux systems. For Windows users, please refer to the Hadoop tutorial first.

Prerequisites

• Java: Ensure you have Java 8 or later installed. You can download it from Oracle's official website.
• Python: Ensure you have Python 3.6 or later installed. You can download it from Python's official website.

Installing SparkNLP

1. Install Apache Spark:

   pip install pyspark

2. Install SparkNLP:

   pip install spark-nlp

3. Verify Installation:

   import sparknlp
   print(sparknlp.version())

Installing Hadoop

1. Download Hadoop:

   • Download the latest stable release from the Apache Hadoop website.

2. Extract the Downloaded File:

   tar -xzvf hadoop-x.y.z.tar.gz

3. Set Up Environment Variables: Add the following lines to your .bashrc or .zshrc file:

   export HADOOP_HOME=/path/to/hadoop
   export PATH=$PATH:$HADOOP_HOME/bin

4. Verify Installation:

   hadoop version

For more detailed instructions, refer to the Hadoop Installation Guide.

Contributions

Thanks for the generous help from @rongliyuan, this project couldn't have been done without you. And thanks to @LagShaggy for useful software suggestions.