Launching steps

The workflow engine launches Steps (Jobs Instances) in response to Events (messages) sent to its WorkflowEngine.handle_message() method. These messages are either a WorkflowMessage START message or a PodMessage that is sent when an existing step has finished (successfully or not).

Starting a workflow

Workflows are started when a client (user) invokes the /workflow/{id}/run method in the DM API. The DM will respond my creating a RunningWorkflow record and then send a WorkflowMessage message to the engine's handle_message() method with action="START".

The message will provide the UUID of a RunningWorkflow record that has been created by the DM. The engine must use this message property to obtain the database record and that of the Workflow. The engine does this using the Workflow API given to the class initialiser. The engine will also use the InstanceLauncher, also given to the engine's class initialiser.

For the simple 2.0 linear engine implementation the engine assumes that the first step declared in the workflow definition is the first step to execute.

As well as identifying the first step, one of the most important parts of the engine's responsibilities is the formation of a dictionary of variables that it needs to pass to the InstanceLauncher in order to actually start the step. The engine must provide names and values for all the variables for every Step that it executes. Each step declares these in its "plumbing" declarations.

As this is a complex process, and a process common to the actions required when continuing a workflow (see below), not just starting it, it is documented separately in Setting step variables.

When starting a workflow's first step the engine must do the following: -

1. Determine the step's variables and values (see Setting step variables)
2. Optionally use the DM Job Decoder to verify that the chosen variables and values satisfy the Step's Job command
3. Create and populate a LaunchParameters object
4. Provide a value for the LaunchParameters running_workflow_step_inputs property. This is the list of step variables in the plumbing block that are marked as from-project
5. Call the InstanceLauncher launch() method, providing the LaunchParameters object

Continuing a workflow

The engine decides whether a workflow is to continue (more steps are the be launched) when a PodMessage message is received by the engine's handle_message() method.

1. The PodMessage contains an instance reference that the engine MUST use to obtain the Instance record from the DM (using the workflow DM API)
2. The Instance record will contain a running_workflow_step_id, a UUID for the RunningWorkflowStep record that is the cause of the instance. The engine MUST retrieve the RunningWorkflowStep (and then RunningWorkflow and Workflow records).
3. Armed with the suite of workflow database records the engine should be able to determine which Step* should be launched next. If there are no more steps the engine should apply the actions described in Terminating a workflow
4. If there is a new step to launch, apply the actions from the Starting a workflow section above.

Stopping a workflow

Workflows that are running can be stopped before they finish when a client (user) invokes the /workflow/{id}/stop method in the DM API. The DM will respond my sending a WorkflowMessage message to the engine's handle_message() method with action="STOP".

In response to this message the engine MUST: -

1. Call the API adapter's set_running_workflow_done() method setting success=False, provide a non-zero value to error_num and an error_msg value (typically User stopped).

Additionally, once the message has been handled the engine must handle the termination of any existing running Steps, knowing that the user has asked the workflow to stop. When steps for the workflow end (with the receoption of a corresponding PodMessage) the engine MUST mark the step's RunningWorkflowStep record (using the DM workflow API) as finished (successfully or otherwise).

The engine MUST NOT launch any new Steps for a Workflow that has been stopped.