Trigger run API configuration

[oheger-bosch]

Problem

The API to trigger a run is very technical and in its raw form hardly usable by end users. For instance, it requires the configuration of things like URLs and credentials of external systems, or paths to resources like Evaluator rules, configuration files, or Reporter templates. End users typically should not need to know and even not be able to change such things.

The proposal about the Plugin Config Manager addresses a major part of this problem by defining data structures to describe the options supported by ORT plugins and set default values for them. Based on this, ORT Server admins can define configurations for the single plugins and restrict the settings that need to be entered by users. This can simplify the API for end users significantly. Nevertheless, this approach still leaves some problems unsolved.

First, not all complex parts of the run configuration are handled by plugins. One example is the Evaluator which also needs to be configured with a valid rules file and a license classification.

Then, there are use cases in which configuration should be adapted dynamically based on convenient user input. For instance, in our setup we have to support the following use cases:

• Users can provide an "application type" out of a list of supported types when triggering a run. This type impacts the rules executed by the Evaluator step. (This can be achieved by either selecting the rule set dynamically or adding a label which is then evaluated by the rules.)
• For the Scanner step, users can select between different FossID instances based on a symbolic name. The FossID configuration is then filled out accordingly. If no instance is selected, FossID is not used, but the source code is scanned with ScanCode.
• In a similar way, users can select templates for the generation of reports based on a symbolic name.

This is possible via the generic parameters structure supported by the trigger run API and providing a parameter validation script that evaluates the passed in parameters and applies them to the job configurations. However, this approach - and this is also the case for plugin options - requires another source of documentation to tell the users which options they have and their meaning. The OpenAPI documentation auto-generated from the endpoints is not helpful for the trigger run endpoint (or even confusing), since it describes the generic schema of the request, but not what the user is supposed to pass based on the parameters or plugin configurations.

Proposal

This proposal is about improving the API to trigger a run to address the listed problems. It consists of two parts: introducing parameter descriptions and defining a way how such descriptions can be configured.

Parameter descriptions

The purpose of parameter descriptions is to configure a set of parameters users can specify when invoking the trigger run endpoint. The description of a single parameter shares many properties with the configuration for a plugin option, such as

• the name
• the data type
• a description of this parameter
• a flag whether this parameter is mandatory
• a default value

The main difference between the configuration of a plugin option and a parameter description is that for the former, it is a priori clear how to process it (by propagating it to the options of the corresponding plugin), while a parameter can require arbitrary logic to modify the job configurations. This needs to be handled in form of a script; when defining a parameter, the definition needs to contain a function that determines the effect of the parameter on the job configurations of the run. So, the function signature will be something like

  (JobConfigurations, String) -> JobConfigurations

The function is passed the current job configurations and the value of the parameter. It yields a modified job configurations object.

With such parameter descriptions in place, similar mechanisms can be implemented as for plugin options:

• The UI to trigger a run can evaluate the parameter descriptions and render a form to enter them in a safe way.
• The endpoint can perform an automatic validation of the parameters.
• It can also apply the parameters to the job configurations automatically.

Trigger run API configuration

The idea behind the trigger run API configuration is to customize the low-level and technical API to trigger a run to be used in a more convenient way by the users of a specific ORT Server installation. The configuration consists of the configuration for the plugin options and the configuration of the parameters available to end users.

Configuration DSL

While the configuration of plugin options is more or less straight-forward, it is more complex for parameters. This is due to their dynamic nature; the logic how a parameter affects job configurations has to be defined somehow. This cannot be done easily in an admin UI.

This proposal therefore suggests moving the configuration to a script and providing a DSL that is tailored towards this purpose. Here is an example how this could look like - of course, the details need to be worked out, just to get a rough impression:

runConfiguration {
    parameter(
        name = "ruleSet",
        description = "Defines the rule set for the Evaluator",
        type = PluginOptionType.STRING_LIST,
        defaultValue = "default",
        allowedValues = listOf("default", "no-oss")
    ) { (jobConfigs, value) ->
          val ruleSetUri = when(value) { ... }
          jobConfigs.copy(
              evaluator = jobConfigs.evaluator?.let { it.copy(ruleSet = ruleSet.Uri) }
          )
    }
    
    ...
    
    reporter {
        plugin("PdfReporter") {
            pluginOption(
                name = "format",
                defaultValue = "...",
            ).restrict(organizationNames = "Org_X")
        }
    }
}

Executing this script yields a configuration for the trigger run endpoint. The script is a replacement for the parameter validation script used so far. It provides a more structured and controlled way to customize the API while still offering full flexibility (by allowing logic in script form).

Having the configuration as a script has the additional benefit that GitOps best practices can be used to maintain it. Changes on the trigger run configuration can be dangerous, since they have the potential to break the API for all users. I would therefore feel uneasy to have an admin UI for it; one wrong click could have dramatic consequences. If a UI for this configuration was offered, there should be facilities to audit log and revert changes. Alternatively, a generator UI could be provided that can generate the basic structure of the configuration script.

Dynamic OpenAPI documentation

If the configuration of the trigger run endpoint is available in a machine-readable form, it can be used for various purposes. One of those could be to create a custom OpenAPI documentation for the endpoint. This is just an idea, not sure whether it is possible to integrate this with the automatically generated documentation. However, this would fix the gap between the documentation and the properties users can actually set when invoking the endpoint.

Summary

This proposal is about

• Extending the configuration of plugin options by a configuration of parameters to trigger a run.
• Defining the configuration in a script with the help of a DSL.