Proposal: Plugin Config Manager

[mnonnenmacher]

Problem

The API currently allows to configure all plugin options freely when creating a new ORT run.
This is problematic, because in a production environment some plugin options should only be configurable by admins of the server.
For example, several plugins allow to configure server URLs for remote services (e.g., server URL for the VulernableCode advice provider), the admins of a server instance should be able to control which URLs are allowed to be used.
Such an enforcement can currently be implemented in the parameter validation script, but this approach does not allow to manage or expose such a configuration in the UI.

Also, the API does currently not provide any information about the available plugin configuration options.
This would be required to be able to dynamically create forms to configure plugins, otherwise all plugin options have to be kept up-to-date in the UI manually.
Thanks to ORT's new plugin API, ORT itself now provides this information, but the server does not yet make use of it.

Another problem is that the UI does currently only support a very limited set of plugin options when creating a new ORT run.

Finally, the admins of a server instance do not only need to set certain plugin options globally, but also need to be able to configure them per organization.
For example, to control which users are allowed to use a specific FossID instance.

Proposal

Add a plugin manager that allows admins to pre-configure plugins, and to control which organizations can use which plugin configurations.

• By default all plugins are available to all organizations and are freely configurable.
• For each plugin, admins can create (multiple) configuration templates where they can set default values for options, and set if users can overwrite them.
• For each plugin configuration template, admins can configure if they are globally available or only for some organizations.
• For each plugin, admins can disable them globally.

Use Cases

Show available plugins

The admin UI should contain a screen that shows all available plugins, grouped by plugin type.

Disable a plugin globally

The plugin screen has a switch for each plugin to disable it globally.
If a plugin is disabled, existing templates for that plugin are not deleted, in case the plugin is later enabled again.

Show plugin templates

The plugin screen has a button for each plugin that leads to a another screen where existing templates for that plugin are shown.

Create a plugin template

The template screen has a form to add a new plugin template.
The form has a switch to set if the template is globally available.
For each plugin option, the form has an input to set the default value and a switch to set if users can override the default value.

Assign templates to organizations

The template screen also shows for each template if it is globally available and if not, which organizations can use it.
If a template is not globally available, there is an option to set which organizations can use it.
For each plugin, only one template can be assigned to an organization.

Delete template

For each template, the screen has a button to delete it.

Request available plugins

For each organization, there is an endpoint that returns all available plugins and their available configuration options.

Trigger ORT run

When triggering an ORT run via the UI, the UI should show forms to configure the plugins for each job.
These forms should only show available plugins and available plugin option.
Any default values for plugin options should be prefilled.

When triggering an ORT run via the API, the request should fail if any plugins or plugin options are set that are not available within the organization.
The error message should clearly state the reason for the error.

Validation

When updating the ORT dependency in the ORT Server, it can happen that the available plugins or plugin options have changed.
This should be handled leniently by plugin configuration templates:
If a template contains a plugin option that does not exist anymore, it should be ignored instead of failing the run.
The plugin management UI should show a warning if templates contain any invalid configuration.

Analyzer

Analyzer plugins configurations require special handling, because they can also be set in the .ort.yml file.
Also, analyzer plugins can be enabled or disabled via the enabledPackageManagers and disabledPackageManagers options.
Any plugin configuration that conflict with the plugin configuration of the organization should fail in the same way, as if the plugins were configured as part of the analyzer job configuration.

Other job configurations

There are other job configurations which should also be configurable only by admins, for example, the evalutor job currently allows to freely chose the paths of license classification or copyright garbage files. This probably requires a similar approach, but is out of scope for this proposal.

Notifier

The ORT notifier does not support plugins, therefore the mechanism described above cannot be used for the notifier job configuration.
Still, the job config contains values like the mail server which should only be settable by admins.
On the long term, the notifier job should be replaced with an event based notification mechanism that was not possible to implement with plain ORT.
Until then, all notifier options that should not be available to users should be removed from the notifier job config.
Instead, admins should be able to configure those options globally.
An exception to that is, for example, the list of email addresses to send notifications to.
This should still be configurable by the user in the notifier job configuration.

[sschuberth]

For each plugin configuration template, admins can configure if they are globally available or only for some organizations.

To clarify, I believe instead of "available" we should say "configure whether they apply to only some or all organizations" or so.

[oheger-bosch]

This is a very detailed proposal for dealing with plugins, but before further proceeding here, I really would like to get a better understanding of the general concept how infrastructure is configured in ORT Server and made available to end users in organizations. Here, "infrastructure" is not limited to plugins, but also includes other things that can be configured, e.g. Evaluator rules, reporter templates, etc. I think, we need to clarify the following questions:

• Which options do admins have to define infrastructure?
• How can end users select the infrastructure they want to use for a run? (Or what will the API to trigger a run look like exactly?)
• Which role does the config validation script play?

There is one more point, I think that is important: Having a UI for the configuration is certainly nice to get an overview over the current options and to do simple changes. However, when it comes to operating a production system, it is best practice to have some kind of versioning to make sure that changes with undesired effects can be reverted easily or to recover the system after a new installation. Therefore, we should probably support some GitOps workflow.

[MarcelBochtler]

I think the most important part of this proposal is that it makes the available configuration options explicit.
The API will fail if a user tries to configure something they are not allowed to do, while the UI won't even show the option to the user.

This is a huge improvement to the current workaround by limiting configurations using a validation script, which rewrites API requests without letting the user know.
I think that this proposal outlines a better, explicit and more elegant way of limiting user configurations.
Therefore, I am strongly in favor of implementing this proposal.

Doing this for plugins first, is a sensible starting point, while other configurations may be migrated to a similar approach later.

[nnobelis]

As the plugins can also be configurated in the customer repositories, in the .ort.yml file (e.g. https://github.com/nordic-institute/harmony-access-point/blob/e9b470bdb0f0c6bc4a80aa92243376b4dea56ffa/.ort.yml#L6), should ORT have a possibility to inhibit this plugin configuration ?

[mnonnenmacher]

Sorry, I don't understand the question. Do you mean that the ORT Server should use the package manager configurations from the .ort.yml file? This is already the case, see the "analyzer" section in the proposal.

[nnobelis]

I see, thanks.