Concept

The Workflow Engine relates to the logic responsible for the interpretation and execution of YAML-based workflow definitions. The engine's repository has two distinct responsibilities: the definition of the workflow schema (a decoder) and the implementation of the workflow state machine (a workflow engine).

Original design requirements: -

A little like our Job Testing framework (Jote) it was considered desirable if...

1. The code would be external to the Data Manager (DM)
2. Testable without the need of kubernetes

These requirements, although appearing to be of great benefit to developers of workflows, introduce significant design considerations.

Running Jobs outside of the DM is a relatively simple task - Jote creates a docker-compose file and launches a container image. But Job instances do not have a complex state (as workflow would undoubtably have).