Linkerd Add-On Model

[Pothulapati]

Present

Currently, Add-Ons are separate charts under charts/linkerd2/add-ons and are marked as dependencies for the linkerd2 charts in requirements.yaml which makes the Helm install possible (as sub-charts are also loaded and generated first). As components in sub-charts also would want to inject proxy containers, they further take a dependency on partials charts. This is done like this instead of having a inject annotation because the proxy injector and add-ons are installed at the same time, and this could cause the injection to fail.

For the CLI to support add-on installation, we use AddOn interface and add-ons have to implement that, which they do in the same folder. and these add-on types (most of which are map[string]interface{} so that we don't have to add a go field every time we add a new configuration field in yaml and the relevant use-case in helm templates, Example: #4761. But they can be concrete types too ) are part of the Values struct which is loaded from linkerd2/values.yaml

Regarding the format of values.yaml, With Helm, Everything under global is passed to all charts making proxy container, etc possible for add-ons. And Values under <addon-name>: are passed to sub-charts thus allowing the default values to be overridden automatically. Default values are under ./charts/add-ons/<add-on>/values.yaml.

With the CLI, The same mechanism is done by passing the Values, and add-on specific field to the chart during generation here.

As once add-ons are installed, we need a way to store the overridden configuration for upgrades, We use the linkerd-config-addons for this and each add-on gets its on field and the overridden configuration is directly placed here. There are also fields under global here which have fields like prometheusUrl, etc which are used to define a external prom instance when linkerd-prometheus is disabled.

linkerd-config-addons is also used for the check command to perform checks based on the add-on configuration. These checks can be found here.

Workflows

With this above context, Let's understand some work-flows

Install

• User can use the --addon-config flag, to pass add-on configuration to the install command. Example Configuration
• linkerd2 chart is loaded normally, and add-on charts are loaded at this point here. parseAddOnValues() is used to get only the enabled add-ons. And these charts are rendered later in the same flow.

Upgrade

• In the upgrade flow, we get the configmap and merge that into Values struct here
• As users would want to update configuration during upgrades, they can continue to use --addon-config to do the same and this configuration is merged here

Users can also skip getting getting the configmap by using --addon-overwrite which means the cm is skipped and going to the default, or to some configuration when combined with --addon-config.

Adding a new Add-On

• Create a new sub-chart under /charts/add-ons/ and add it as a dependency to linkerd2 chart.
• Create a new struct and implement the Add-On Interface under pkg/charts/, and this field should be a sub-field under Values struct.
• Update parseAddOnValues to know about the add-on and add it to the return list if its enabled. _(This could've been automated and was later removed as it used reflection and was confusing.).
• Update linkerd-config-addons to print the new add-on field.

Problems with the current flow

• Currently, Configuration can be shared between linkerd2 and sub-charts, using global, and vice-versa by using the <add-on> field in values.yaml but there is no way for add-ons to share configuration details between them i.e Values under prometheus: are not visible in the grafana chart. This was pretty important for the dashboard add-on as it needs to know if grafana and jaeger are present, This was solved by mounting linkerd-config-addons into linkerd-web but a more general approach would be useful.
• linkerd-config-addons is rendered differently in Helm and CLI. In Helm, the default values are merged and printed. But as Helm does not use linkerd-config-addons for upgrades, This did not make any difference.

Pull Requests

Tracing

• linkerd/linkerd2#4407 - Add HealthChecks for Tracing Add-On
• linkerd/linkerd2#4139 - Tracing Add-On Flags in Readme
• linkerd/linkerd2#3955 - Tracing Add-on For Linkerd

Grafana

• linkerd/linkerd2#4381 - Add global.grafanaUrl for linking existing grafana use-case
• linkerd/linkerd2#4321 - Add health checks for grafana add-on
• linkerd/linkerd2#4320 - Move grafana templates into a separate sub-chart as a add-on
• linkerd/linkerd2#4319 - Make Grafana optional in linkerd-web
• linkerd/linkerd2#3979 - Make Grafana a soft requirement

Prometheus

• linkerd/linkerd2#4756 - prometheus: add add-on checks
• linkerd/linkerd2#4724 - Remove/Relax prometheus related checks
• linkerd/linkerd2#4390 - Add global.prometheusUrl field for byop use-case
• linkerd/linkerd2#4362 - Move Prometheus as an Add-On

Miscellaneous

• linkerd/linkerd2#4682 - Use add-on level values.yaml for chart generation
• linkerd/linkerd2#4666 - cli: support url with addon-config flag
• linkerd/linkerd2#4377 - Add addon-overwrite flag
• linkerd/linkerd2#4318 - Refactor linkerd-config-addons configmap
• linkerd/linkerd2#4271 - Support Multi-stage install with Add-On's
• linkerd/linkerd2#4247 - Refactor AddOn Installation