Background:

Auron currently does not support Spark 4.0.1. With the release of Spark 4.0.1 on September 6, 2025, there are significant changes that need to be addressed in Auron to ensure compatibility. Spark 4.0.1 requires compilation with JDK 17 or higher, and it is pre-built with Scala 2.13, with official support for Scala 2.12 now dropped.

This issue aims to track the necessary updates for Auron to support Spark 4.0.1, including adjustments to the API, SQLContext, SparkSession handling, and dependency compatibility.

Key Tasks:

• API Compatibility: Update Auron’s code to accommodate new and deprecated APIs introduced in Spark 4.0.1.
• SQLContext and SparkSession: Refactor code to ensure compatibility with changes in how Spark 4.0 handles SQLContext and SparkSession.
• Dependency Updates: Ensure that all project dependencies are compatible with Spark 4.0.1, especially considering the removal of support for Scala 2.12.
• Testing: Modify and add tests to ensure that Auron works seamlessly with Spark 4.0.1, verifying both performance and feature compatibility.