Stack Deployment: Generate env files from secret vaults (Vault / Infisical)

[radhakrisri]

Is your feature request related to a problem? Please describe

Bringing up applications like authentik (https://docs.goauthentik.io/docs/installation/docker-compose), plane.so (https://docs.plane.so/docker-compose) etc... involves the following process:

1. Download docker-compose.yml from a repository
2. Configure a .env file
3. Launch.

This process is applicable to a significant number of applications, not just both above.

Managing these modified .env files for a number of services usually involves version controlling them in git. More flexible schemes involve managing them in secret vaults like Hashicorp Vault or Infisical.

Typically these secret vault solutions provide mechanisms to export the secrets into a .env format.

However, one needs to stitch up a different workflow/ script to generate these .env files and then deploy the docker-compose.yml as a stack in portainer.

Describe the solution you'd like

A solution like this streamlines the entire deployment process and provides a much more automated and robust mechanism:

1. Stack Deployment UI in Portainer provides two configuration options:
   1. Source for docker-compose.yml. Options: (i) Downloadable URL or (ii) Git Repository
   2. A configuration for a secret vault. Options: (i) Hashicorp Vault or (ii) Infisical
2. When the user hits deploy, Portainer does the following:
   1. Fetch docker-compose.yml (wget/ curl)
   2. Prepares a .env (or stack.env) by using vault specific mechanisms
   3. Then launches docker compose.

This provides a streamlined workflow for deploying stacks in Portainer.

1. Ease of deployment: No additional workflows need to be scripted.
2. Centralized management of environments: Secret vaults (Hashicorp Vault/ Infisical) can be used to manage different environments (dev/ staging/ production) while same deployment mechanisms can be used with Portainer.

Describe alternatives you've considered

Alternatives I've been using is to maintain a script that talks to Portainer API and Infiscial API to orchestrate generate of .env and deploying stacks with portainer.

Additional context

None

[Adamou02]

Any updates on this since 2024? I'm planning to work on it myself in a local branch. Anyone interested in doing it with me?