Replace Spark 2.4 with Spark 3 support

[benedeki]

Background

Spark 2.4 is really outdated. Let's focus on the more current versions of Spark

Feature

Switch the library to work with Spark 3.
The crossbuild can then focus on Scala 2.12 and Scala 2.13.

Consider switching the build from Java 1.8 to Java 11.