Accessing files

Workflow Steps are simply Jobs. Steps, like Jobs, run is a subdirectory of a Project. Neither Jobs nor Steps have general access to Project files.

If a step needs to access Project files, or files generated by other steps it declares this need in the step's Plumbing.

Accessing project files

If a Step needs to access a Project file it must make two declarations.

The Workflow must declare an input Workflow Variable. When a user runs the Workflow they are required to provide a value for the input - a file in the Project volume. This is enforced by the DM, the Workflow Engine can assume that all input variables have been declared. Secondly, any Step that wishes to use this file needs to declare it in their plumbing.

Here's an example of a Workflow inout variable: -

variables:
  inputs:
    type: object
    properties:
      candidateMolecules:
        title: Molecules
        type: file

When the user runs the workflow they will be required to provide a value for the variable candidateMolecules, a file in the Project.

How does a Step use that file? Files are not presented to every step, instead, if a step wants to access a project file it must declare this in its Plumbing, illustrated below.

step:
- name: step-1
  plumbing:
  - variable: inputFile
    from-workflow:
      variable: candidateMolecules

This reveals several important facts about the steps: -

1. The step's Job has a variable called inputFile
2. The step expects its inputFile value to be set the the value of the workflow variable candidateMolecules

Who puts the file in the Steps's instance directory?

The DM does that when the workflow engine launches the step, with help from the engine, which passes the values of the selected workflow files using the InstanceLauncher launch() command's LaunchParameters object. The DM ensures that the named files are hard-linked into the step's instance directory.

Accessing files in prior steps