Workflow validation

There are three levels of validation. They correspond to the actions of "creating", "tagging", and "running" a workflow, each requiring a slightly more invasive and detailed inspection of the YAML file, something the schema is not able to do entirely without help. The validator is used by the DM prior to the execution of workflows, then engine can therefore assume that when a workflow is run validation will have been performed and will have been successful.

The WorkflowValidator module exposes the level and validation results as an Enum and dataclass: -

class ValidationLevel(Enum):
"""Workflow validation levels."""

    CREATE = 1
    RUN = 2
    TAG = 3


@dataclass
class ValidationResult:
    """Workflow validation results."""

    error_num: int
    error_msg: list[str] | None

The ValidationResult indicates success with an instance of the class with an error_num of 0 (zero) and where error_msg is None. On error the error_num will be non-zero and error_msg will contain at least one user-digestible error message.

ValidationResult SHOULD indicate success with an instance of the class with an error value of 0 and where error_msg is None. On error the error MUST be non-zero and error_msg MUST contain at least one user-digestible error message.

The WorkflowValidator also exposes a validate() method: -

class WorkflowValidator:

    @classmethod
    def validate(
        cls,
        *,
        level: ValidationLevel,
        workflow_definition: dict[str, Any],
        variables: dict[str, Any] | None = None,
    ) -> ValidationResult:

Validation will improve and become more invasive as we develop the engine.

CREATE validation

This is the simplest level of validation and simply ensures that the provided workflow definition complies with the schema.

RUN validation

As well as running the CREATE validation stage this level is responsible for ensuring that the workflow can be run. The logic should make sure that the workflow represents something that can be run by: -

1. Checking all the required workflow variables have been provided

TAG validation

As well as running the RUN validation stage this level is responsible for ensuring that the workflow can be tagged: -

1. Checking the step names are unique (within the workflow)
2. There are no duplicate output variable names
3. Workflow variable names are unique
4. replicating steps refer to an exiting input variable

See also

1. The Validation and starting workflows (WorkflowValidator) section of the Shortcut doc Workflow engine design