feat(envs): add configurations to include extra envs for containers

[tthvo]

Fixes #203

Descriptions

Adding an field storage.config.extra.envVars and storage.config.extra.envSources to allow users to configure additional environment variables for the storage container.

This can be extended to other containers (i.e. core, database, etc).

[andrewazores]

Looks great to me.

This can be extended to other containers (i.e. core, database, etc).

What are your thoughts on how to handle multi-container Pods?

[tthvo]

What are your thoughts on how to handle multi-container Pods?

I guess we are all well set up to handle that since we have separate helm value section for each container (i.e. core, db, storage, grafana, datasource, oauth2Proxy and openshiftOauthProxy). So, each can have a section, for example:

core:
  config:
    extra:
      enVars: []
      envSources: []

grafana:
  config:
    extra:
      enVars: []
      envSources: []

In the future, if we are adding any new container, its customization probably is going to have its own section and follows the same pattern, right?

[andrewazores]

Makes sense, but I do see one spot where it might get funny: the auth proxy container can be used in more than one Deployment. Already it can be present in both the main Cryostat Pod and the Reports Pod. I don't know what kind of additional configuration the user might want to set on this container, but with this structure it would only be possible to give both instances the same configurations, which might not be suitable.

(It would also be nice if there could be someway to reuse these snippets in different places and not copy-paste the same structures everywhere, but I think this is a Helm/yaml limitation)

[tthvo]

(It would also be nice if there could be someway to reuse these snippets in different places and not copy-paste the same structures everywhere, but I think this is a Helm/yaml limitation)

Unfortunately, we need to copy-paste 😅 I guess we can generate the values.yaml from a template file, let's say values.yaml.tpl with go/template. But that sounds a bit over-engineering :D

Makes sense, but I do see one spot where it might get funny: the auth proxy container can be used in more than one Deployment. Already it can be present in both the main Cryostat Pod and the Reports Pod. I don't know what kind of additional configuration the user might want to set on this container, but with this structure it would only be possible to give both instances the same configurations, which might not be suitable.

Right, totally unaware of that 😅 I guess we can do the following where we have a common settings and additional settings for specific pod use case. This is similar to how k8s defines SecurityContext.

The settings with pod-specific context will take precedence. This works well because env var that are defined later takes precendence [0].

oauth2Proxy:
  config:
    ## @extra oauth2Proxy.config.extra Extra configurations for the OAuth2 Proxy container
    extra:
      ## @param oauth2Proxy.config.extra.envVars [array] Extra environment variables for the OAuth2 Proxy container. See: [Define Environment Variables](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/)
      envVars: []
      ## @param oauth2Proxy.config.extra.envSources [array] Sources for extra variables for the OAuth2 Proxy container. See: [Define Environment Variables From ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#configure-all-key-value-pairs-in-a-configmap-as-container-environment-variables)
      envSources: []
    podContext: # This probably can be renamed
      main:
        ## @extra oauth2Proxy.config.podContext.main.extra Extra configurations for the OAuth2 Proxy container in main pod. This will overwrite common settings in oauth2Proxy.config.extra
        extra:
          envVars: []
          envSources: []
      report:
        ## @extra oauth2Proxy.config.podContext.report.extra Extra configurations for the OAuth2 Proxy container in report pod. This will overwrite common settings in oauth2Proxy.config.extra
        extra:
          envVars: []
          envSources: []

[tthvo]

Or maybe this would be a better value settings (looks simpler). I am fine either way for as long as we document it. These values can be set in proxy templates _oauth2Proxy.tpl and _reports_authproxy.tpl, where common settings are placed first before specific pod overwrites. WDYT?

oauth2Proxy:
  config:
    ## @extra oauth2Proxy.config.extra Extra configurations for the OAuth2 Proxy container
    extra:
      ## @param oauth2Proxy.config.extra.envVars [array] Extra environment variables for the OAuth2 Proxy container. See: [Define Environment Variables](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/)
      envVars: []
      ## @param oauth2Proxy.config.extra.envSources [array] Sources for extra variables for the OAuth2 Proxy container. See: [Define Environment Variables From ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#configure-all-key-value-pairs-in-a-configmap-as-container-environment-variables)
      envSources: []
      podContext:
        main:
            envVars: []
            envSources: []
        report:
          envVars: []
          envSources: []

[andrewazores]

I like that second style better.

And I agree, generating the values.yaml from a template sounds like it's getting into over-engineering. It might be an idea worth keeping in mind in case the configuration keeps getting more complex, though, since we already do have a lot of duplicated template snippets across the various components.

[tthvo]

yea haha I wish helm figures something out about this :D but sounds good! I'll circle back near end of week for follow-up PRs!

[tthvo]

Hey @andrewazores, this should be ready for reviews 😄

Instead of PRs, I separate the changes in separate commits for easy reviews. All containers now support extra env vars and env sources. PTAL!

[andrewazores]

Thanks @tthvo !