Workflow validation

There are three levels of validation. They correspond to the actions of "creating", "tagging", and "running" a workflow, each requiring a slightly more invasive and detailed inspection of the YAML file, something the schema is not able to do entirely without help. The validator is used by the DM prior to the execution of workflows, then engine can therefore assume that when a workflow is run validation will have been performed and will have been successful.

The WorkflowValidator module exposes the level and validation results as an Enum and dataclass: -

class ValidationLevel(Enum):
"""Workflow validation levels."""

    CREATE = 1
    RUN = 2
    TAG = 3


@dataclass
class ValidationResult:
    """Workflow validation results."""

    error_num: int
    error_msg: list[str] | None

The ValidationResult indicates success with an instance of the class with an error_num of 0 (zero) and where error_msg is None. On error the error_num will be non-zero and error_msg will contain at least one user-digestible error message.

ValidationResult will indicate success with an error_num value of 0. The error message is not defined unless there's an error. When there is an error the error_nun will be a non-zero value and error_msg will contain at least one user-digestible error message.

The WorkflowValidator also exposes a validate() method: -

class WorkflowValidator:

    @classmethod
    def validate(
        cls,
        *,
        level: ValidationLevel,
        workflow_definition: dict[str, Any],
        variables: dict[str, Any] | None = None,
    ) -> ValidationResult:

Validation will improve and become more invasive as we develop the engine.

Validation

There are three levels of validation, used when a workflow is created, when it is tagged, and when it is run. The three increasing levels of validation are described below: -

CREATE

This is the simplest level of validation and simply ensures that the provided workflow definition complies with the schema.

The DM will save workflow definitions even if they fail this level of validation. It does this so the Workflow database records can be used as a persistent storage facility. User of the workflow API must therefore expect to find workflow definitions that do not comply with the schema.

Workflows cannot be tagged, or run unless they pass this basic level of validation.

TAG

As well as running the CREATE validation stage this level is responsible for ensuring that the workflow can be tagged: -

1. Checking the step names are unique (within the workflow)

2. There are no duplicate output variable names

3. Workflow variable names are unique

4. replicating steps refer to an exiting input variable

   The function is a WIP

RUN

As well as running the TAG validation stage this level is responsible for ensuring that the workflow can be run. The logic should make sure that the workflow represents something that can be run by: -

1. Checking all the required workflow variables have been provided

   The function is a WIP

See also

1. The Validation and starting workflows (WorkflowValidator) section of the Shortcut doc Workflow engine design