Scattersphere: Job Coordination API for Tasks

Scattersphere is a lightweight job coordination API designed to run a simple DAG of tasks.

It draws inspiration from many projects on Github such as Orchestra, Chronos, and the like. It is designed to be self-contained, and extensible.

Prerequisites

• Java 11 SDK or higher (LTS version recommended)
• Maven 3.6+ (for building)

Getting Started

There is a maintained Wiki here. You can also view sample code in the test suite here.

Dependencies

• Scala 2.13.12
• Spark 3.5.3

And that's all

NOTE: Spark is only required if you plan on coordinating jobs using the Spark framework. If you don't, you can just use Scattersphere in a single server setup. Scattersphere will be run as a server on distributed servers eventually.

Contributing

Please read CONTRIBUTING.md

Building

See BUILDING.md