Setting step variables

Setting the variable map (a Python dictionary of Step variable names and values) is probably the most complex part of workflow execution. Steps (Jobs) are launched using the InstanceLauncher class object that is provided when the engine is initialised. The launch() method of this object takes a LaunchParameters object where variables can either be provided in the specification or separately using the launch parameters specification_variables member: -

# Provide variables in the launch parameters specification
launch_parameters.specificaion["variables"] = step_variables
# ...or specification_variables.
# These variables "replace" any specification variables.
launch_parameters. specification_variables = step_variables

With some examples, we'll now explore the process of collecting together a step's variables, which involves inspecting the RunningWorkflow record, the step's input, options and outputs, and the workflow's variable-mapping.

Imortantly: --

• All top-level variables defined by the workflow are provided by the User when they run the workflow
• The variables are available in the RunningWorkflow record's variables property (a map of names and values)
• The engine can assume that the workflow has been validated and all the required variables have been provided

In the following superfluous properties are omitted from the workflow for brevity. The following are essentially excerpts from otherwise valid workflows.

Example 1

Here, step variables are provisioned from workflow variables (present in the RunningWorkflow record).

If we have the following Workflow: -

variable-mapping:
  inputs:
  - name: candidateMolecules
  outputs:
  - name: clusteredMolecules
    from:
      step: final-step
      output: outputFile
  options:
  - name: rdkitPropertyName
    as:
    - option: name
      step: step1
  - name: rdkitPropertyValue
    as:
    - option: value
      step: step1

steps:
- name: step-1
  inputs:
  - input: inputFile
    from:
      workflow-input: candidateMolecules
  outputs:
  - output: outputFile
    as: first-step.out.smi

And the following RunningWorkflow variables (note that all values are strings): -

{
  "candidateMolecules": "C=1CCCCC1",
  "rdkitPropertyName": "x",
  "rdkitPropertyValue": "17"
}

Then the step variables will be: -

{
  "inputFile": "C=1CCCCC1",
  "outputFile": "first-step.out.smi",
  "name": "x",
  "value": "17"
}

• step-1's inputFile is derived from the workflow's input candidateMolecules
• The variable mapping also declares two options for step-1 (name and value)

Example 2